Quo Vadis, World Modeling?
Quick Answer
This paper introduces Agent-Centric Interactive World Proxies, enhancing world modeling by focusing on agent-usable information transitions rather than just physical state predictions.
Quick Take
It categorizes world proxies into six functional forms and outlines their roles in improving agent performance across three levels: inference-time guidance, training-time optimization, and agent-proxy co-evolution.
Key Points
- Introduces six functional forms of world proxies for agent feedback: dynamics, spatial, execution, memory, skill, and reward.
- Proposes a shift from physical state transitions to actionable information transitions for agent improvement.
- Identifies three progressive levels of agent empowerment: inference-time guidance, training-time optimization, and co-evolution.
- Aims to enable agents to plan better, learn faster, and evolve continuously.
- Establishes a roadmap for developing versatile world proxies in AI and robotics.
Paper Resources
📖 Reader Mode
~3 min readAuthors:Yu Yang, Xuemeng Yang, Licheng Wen, Lingdong Kong, Xiaobin Hu, Dongyue Lu, Wei Chow, Xiyan Huang, Yuxiang Feng, Yue Liao, Jianbiao Mei, Daocheng Fu, Rong Wu, Pinlong Cai, Ran Yi, Ying Tai, Jiangning Zhang, Botian Shi, Yong Liu, Shuicheng Yan
Abstract:Continually improving agents require dynamic interaction feedback beyond static supervision, yet direct real-environment interaction is costly, slow, unsafe, and hard to parallelize. World modeling offers a natural intermediate proxy that allows agents to query lower-cost, more controllable feedback before committing to real actions. Classical world models instantiate this proxy primarily through future physical-state prediction, a formulation useful yet narrow for agents that require actionable feedback beyond raw state transitions. In this work, we conceptualize Agent-Centric Interactive World Proxies, shifting the fundamental paradigm from physical state transitions to agent-usable information transitions, such as execution outcomes, retrieved experiences or skills, and verification signals, broadening the scope of world modeling to provide versatile feedback for continually improving agents. To systematically map this design space, we organize world proxies into six functional forms based on their feedback modalities: dynamics, spatial, execution, memory/experience, skill, and reward/verification proxies, which together characterize the primary ways world modeling serves agent improvement. We further analyze how these proxies empower agents across three progressive levels: L.1 Inference-Time Guidance, where proxy outputs enrich in-context information for superior decisions; L.2 Training-Time Optimization, where proxy outputs yield rewards, critiques, or synthetic rollouts for policy learning; and L.3 Agent-Proxy Co-Evolution, where real-environment evidence continuously updates both the proxy and the agent for co-evolution. Ultimately, this work recasts world modeling into an agent-centric paradigm, establishing a roadmap for building world proxies that empower agents to plan better, learn faster, and evolve continually.
| Comments: | Technical Blog at this https URL GitHub Repo at this https URL |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Robotics (cs.RO) |
| Cite as: | arXiv:2608.02713 [cs.CV] |
| (or arXiv:2608.02713v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2608.02713 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Lingdong Kong [view email]
[v1]
Mon, 3 Aug 2026 17:59:58 UTC (4,702 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
ProMoE-FL introduces a Prototype-conditioned Mixture-of-Experts framework for multimodal federated learning, effectively addressing missing modalities. It outperforms existing methods on four chest X-ray datasets, demonstrating superior feature synthesis capabilities in both homogeneous and heterogeneous settings.