MultiView-Bench: A Diagnostic Benchmark for World-Centric Multi-View Integration in VLMs
Quick Answer
MultiView-Bench is a new benchmark for evaluating multi-view integration in Vision-Language Models (VLMs), highlighting their struggles with 3D spatial relations and biases in object perception.
Quick Take
MultiView-Bench is a new benchmark for evaluating multi-view integration in (VLMs), highlighting their struggles with 3D spatial relations and biases in object perception. The proposed ViewNavigator framework enhances model performance by 3-5x on this benchmark, addressing the limitations of existing VLMs in holistic 3D scene comprehension.
Key Points
- MultiView-Bench evaluates the ability to integrate observations across multiple viewpoints.
- Existing VLMs perform well on 2D relations but struggle with 3D spatial integration.
- ViewNavigator improves model performance on MultiView-Bench by 3-5 times.
- VLMs show biases related to unconventional axis directions and object textures.
- The benchmark is essential for tasks like mechanical part assembly.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Recent benchmarks for VLMs largely assess single- or limited-view perception, leaving untested the core cognitive ability to integrate observations across viewpoints into a coherent, world-centric (allocentric) 3D mental model. We introduce MultiView-Bench, a diagnostic benchmark expressly designed to evaluate multi-view integration for holistic 3D scene comprehension. Unlike existing datasets that focus on pixel-level mapping or camera-relative navigation, MultiView-Bench requires models to decouple object positioning from transient perspectives and ground them in a fixed global coordinate system. This capability serves as a prerequisite for VLMs before being deployed for downstream tasks such as mechanical part assembly. Our systematic evaluation of frontier VLMs reveals consistent failure modes: strong performance on 2D planar relations from a single image, but marked difficulty with 3D spatial relations and with aggregating information across views. We further identify biases in VLMs, such as struggles with unconventional axis directions and sensitivity to object colorways and texture variations. Acknowledging these limitations, we propose ViewNavigator, a multi-agent framework that actively selects informative viewpoints, perceives, and fuses multi-view evidence, improving diverse base models on MultiView-Bench even under a strict budget-matched comparison (and by 3-5x for the full agent).
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.08970 [cs.CV] |
| (or arXiv:2607.08970v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2607.08970 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Hantao Zhang [view email]
[v1]
Thu, 9 Jul 2026 22:22:42 UTC (13,767 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
ProMoE-FL introduces a Prototype-conditioned Mixture-of-Experts framework for multimodal federated learning, effectively addressing missing modalities. It outperforms existing methods on four chest X-ray datasets, demonstrating superior feature synthesis capabilities in both homogeneous and heterogeneous settings.