SearchEyes: Towards Frontier Multimodal Deep Search Intelligence via Search World Simulation
Quick Answer
SearchEyes introduces a unified framework for multimodal search agents, leveraging a typed knowledge graph and Perception-Knowledge Chains (PKC) to enhance multi-hop reasoning.
Quick Take
SearchEyes introduces a unified framework for multimodal search agents, leveraging a typed knowledge graph and Perception-Knowledge Chains (PKC) to enhance multi-hop reasoning. The model outperforms existing open-source benchmarks, achieving a 6.2-point improvement on average across six knowledge-intensive tasks.
Key Points
- SearchEyes uses a simulated search world to unify training data, environments, and rewards.
- Introduces Perception-Knowledge Chains (PKC) for constrained multi-hop path sampling.
- Achieves state-of-the-art performance among open-source multimodal search agents.
- SearchEyes-27B improves over the strongest baseline by 6.2 points on average.
- Utilizes Hop-Anchored Policy Optimization (HaPO) for efficient credit assignment.
Paper Resources
📖 Reader Mode
~2 min readAuthors:Zhengbo Jiao, Yiming Cheng, Yilei Jiang, Kaituo Feng, Rui Huang, Tianyi Jiang, Juanxi Tian, Jiapeng li, Qunzhong Wang, Tailai Chen, Qianshan Wei, Chuan Xiao, Shanyu Rong, Yangfu Li, Yanhan Zhou, Yunpu Ma, Yifan Zhang, Xiangyu Yue
Abstract:Training multimodal search agents to perform multi-hop reasoning remains challenging due to a fundamental structural disconnect: existing pipelines construct training data, search environments, and reward signals independently, causing synthesized structural metadata to be discarded, environments to rely on irreproducible external engines, and RL rewards to remain sparse at the trajectory level. We present \textbf{SearchEyes}, which uses a typed knowledge graph as the backbone of a \emph{simulated search world} that unifies all three components. We propose \textbf{Perception-Knowledge Chains (PKC)} to sample constrained multi-hop paths over the visual-knowledge intersection of Wikidata5M, retaining hop-level entity metadata that simultaneously defines a self-contained search world and step-level reward anchors. We further propose \textbf{Hop-Anchored Policy Optimization (HaPO)}, which reuses these anchors for step-level credit assignment without a separately trained process reward model. Experiments on six multimodal knowledge-intensive benchmarks show that SearchEyes achieves state-of-the-art performance among open-source multimodal search agents, with SearchEyes-27B improving over the strongest open-source baseline by 6.2 points on average.%
| Comments: | Project page: this https URL |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.05943 [cs.AI] |
| (or arXiv:2607.05943v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2607.05943 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhengbo Jiao [view email]
[v1]
Tue, 7 Jul 2026 07:43:04 UTC (1,872 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →Onnes: A Physics-Grounded LLM Simulator for Cryogenic Fault Diagnosis in Quantum Computing Infrastructure
Onnes is a physics-grounded digital twin simulator for dilution refrigerators, enhancing cryogenic fault diagnosis in quantum computing. It achieves a classification accuracy of 99.0% using few-shot demonstrations, matching a supervised ML classifier, while maintaining a low false alarm rate of 6.4% on real hardware.