Does AI Understand Imaging? A Systematic Benchmark of Agentic AI for Computational Imaging Tasks
Quick Answer
The ImagingBench benchmark evaluates agentic AI models like Gemini, GPT, and Qwen on 20 computational imaging tasks, revealing that these models underperform compared to specialized methods, particularly in computational sensing.
Quick Take
The ImagingBench benchmark evaluates agentic AI models like Gemini, GPT, and Qwen on 20 computational imaging tasks, revealing that these models underperform compared to specialized methods, particularly in computational sensing. Despite generating visually plausible outputs, their fidelity to reference standards is poor, highlighting a significant gap in performance.
Key Points
- ImagingBench includes 20 tasks across five categories in computational imaging.
- Agentic models consistently underperform compared to specialized non-agentic methods.
- Notable weaknesses observed in lensless imaging and time-of-flight imaging tasks.
- Planner guidance offers limited improvement over the fixed-prompt Expert baseline.
- The benchmark highlights the gap between semantic visual competence and physical imaging performance.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Vision-language models (VLMs) and agentic AI have shown strong performance on semantic visual tasks, but it remains unclear whether they can handle the physics and inverse problems that underlie computational imaging. We present ImagingBench, a benchmark of 20 computational imaging tasks spanning five categories: ray and wave optics, image signal processing, inverse reconstruction, computational sensing, and calibration. ImagingBench evaluates three complementary settings: Expert, fixed expert-guided inverse reconstruction; Planner, planner-guided inverse reconstruction; and Forward, forward-system simulation for consistency checking. We benchmark leading proprietary and open-source image-centric multimodal systems, including Gemini, GPT, and Qwen, and compare them with representative task-specific non-agentic baselines. Across tasks, agentic models remain consistently weaker than specialized methods, especially on computational sensing problems such as lensless imaging, event-based reconstruction, time-of-flight imaging, and holography. Planner guidance provides only modest and inconsistent gains over the fixed-prompt Expert baseline. Although the models often generate visually plausible outputs, their reference-based fidelity remains poor, revealing a substantial gap between semantic visual competence and physically grounded imaging performance. ImagingBench provides a unified testbed for measuring this gap and tracking progress in agentic AI for computational imaging.
| Comments: | 14 pages, 11 figures. Preprint / work in progress. Paper Webpage: this https URL |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.07189 [cs.AI] |
| (or arXiv:2607.07189v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2607.07189 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ethan Chung [view email]
[v1]
Wed, 8 Jul 2026 09:23:56 UTC (23,258 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →Onnes: A Physics-Grounded LLM Simulator for Cryogenic Fault Diagnosis in Quantum Computing Infrastructure
Onnes is a physics-grounded digital twin simulator for dilution refrigerators, enhancing cryogenic fault diagnosis in quantum computing. It achieves a classification accuracy of 99.0% using few-shot demonstrations, matching a supervised ML classifier, while maintaining a low false alarm rate of 6.4% on real hardware.