Perceptual 3D Simulation With Physical World Modeling
Quick Answer
P3Sim is a novel physical world modeling system that predicts future scene states from partial observations and incomplete 3D transformations.
Quick Take
P3Sim is a novel physical world modeling system that predicts future scene states from partial observations and incomplete 3D transformations. It integrates a learned world model, geometric conditioning, and persistent memory to enhance generalization across various 3D tasks, including novel view synthesis and dynamic scene prediction. This approach aims to improve 3D scene understanding in robotics and computer vision.
Key Points
- P3Sim combines a learned physical model with geometric conditioning and scene memory.
- It predicts distributions of scene variables based on multimodal inputs.
- The system enables online updates and maintains consistency under uncertainty.
- P3Sim generalizes across diverse 3D transformation tasks effectively.
- Published as a conference paper at CVPR 2026.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Predicting how a scene will evolve after a desired 3D transformation from images is a central goal in vision, graphics, and robotics. Yet unlike ideal simulators with full access to 3D geometry and dynamics, real world systems must rely on perceptual inputs and local actions that are inherently partial and incomplete. In this work, we present P3Sim, a physical world modeling system that simulates future scene states under both partial observations and incomplete 3D transformation signals. P3Sim is composed of three interacting components: a learned physical world model, a geometric conditioning module, and a persistent scene memory. The world model interprets perception as probabilistic inference over multimodal scene variables, providing predictions of the distributions of any scene variable conditioned on any combination of others. The geometric conditioning module provides a partial 3D transform signal for conditioning the world model at inference time. The persistent scene memory integrates predictions over time, enabling online updates and consistency under uncertainty. By combining learned inference with explicit geometric structure, P3Sim balances data-driven flexibility with built-in inductive bias. This design yields a flexible perceptual simulator that generalizes across diverse 3D transformation tasks, such as novel view synthesis, object manipulation, and dynamic scene prediction, advancing toward general purpose 3D scene understanding and transformation.
| Comments: | Published as a conference paper at CVPR 2026 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2606.27575 [cs.CV] |
| (or arXiv:2606.27575v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2606.27575 arXiv-issued DOI via DataCite |
Submission history
From: Wanhee Lee [view email]
[v1]
Thu, 25 Jun 2026 22:01:34 UTC (10,804 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →LLM-Guided ANN Index Optimization for Human-Object Interaction Retrieval
A phase-aware LLM agent optimizes human-object interaction retrieval, outperforming Optuna TPE by 33.3% and VDTuner by 34.2% on the HICO-DET benchmark. This method enhances throughput by 15.3x over UniIR and demonstrates strong transferability across vector database management systems.