Seeing without Looking: Do Vision-Language Benchmarks Really Test Vision?
Quick Take
This study reveals that current vision-language model benchmarks, such as hallucination tests, inadequately assess visual grounding, as significant image token removal only slightly affects performance. The analysis across multiple VLMs shows that predictions are less sensitive to fine-grained visual evidence loss, suggesting a need for improved evaluation methods.
Key Points
- Removing a large fraction of image tokens minimally impacts performance on hallucination benchmarks.
- VLMs show reduced sensitivity to the loss of fine-grained visual evidence.
- Internal support for correct answers weakens even if final predictions remain unchanged.
- Layer-wise analysis indicates increasing similarity among visual tokens in deeper layers.
- Current benchmarks fail to reliably evaluate fine-grained visual grounding.
Article Content
From source RSS / original summaryarXiv:2605. 22903v1 Announce Type: new Abstract: Benchmark accuracy is often implicitly assumed to reflect grounded visual understanding in vision-language models (VLMs), yet it remains unclear to what extent such scores truly reflect reliance on visual evidence. Motivated by a surprising observation that removing a substantial fraction of image tokens only degrades model performance very slightly on a widely used hallucination benchmark, we systematically investigate this mismatch in a set of open-source VLMs.
Our analysis spans multiple levels of granularity, spanning global visual degradation, localized occlusion, question reformulation, answer-space expansion, and decision-level analyses beyond standard accuracy. We further complement these behavioral results with a layer-wise analysis of vision-token geometry. Throughout the experiments, we find that although VLMs do incorporate visual input, their predictions are less sensitive to the loss of fine-grained visual evidence that standard accuracy should have suggested.
Even when the final prediction remains unchanged, the model's internal support for the correct answer may already be weakened. We further complement a representation-level analysis, which shows increasing similarity among visual tokens in deeper layers, providing a possible explanation for our findings. Together, these results suggest that current benchmarks are not sufficient to reliably evaluate fine-grained visual grounding in VLMs.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →Evi-Steer: Learning to Steer Biomedical Vision-Language Models through Efficient and Generalizable Evidential Tuning
Evi-Steer introduces a novel evidential tuning framework for BiomedCLIP, achieving 0.11% parameter updates while enhancing uncertainty-aware fine-tuning. It outperforms state-of-the-art methods across 15 biomedical imaging datasets, proving effective in few-shot learning and domain shifts for clinical applications.