MKG-RAG-Bench: Benchmarking Retrieval in Multimodal Knowledge Graph-Augmented Generation
Quick Answer
MKG-RAG-Bench introduces a new benchmark for evaluating retrieval in multimodal knowledge graph-augmented generation, addressing critical challenges in aligning heterogeneous knowledge across modalities.
Quick Take
MKG--Bench introduces a new benchmark for evaluating retrieval in multimodal knowledge graph-augmented generation, addressing critical challenges in aligning heterogeneous knowledge across modalities. The benchmark, constructed from general and medical domains, highlights the importance of retrieval quality on generation outcomes, emphasizing that effective multimodal retrieval is essential for improving MKG-RAG performance.
Key Points
- MKG-RAG-Bench evaluates retrieval in multimodal knowledge graph-augmented generation.
- Constructed from multimodal knowledge graphs in general and medical domains.
- Highlights retrieval as a critical bottleneck affecting generation outcomes.
- Demonstrates that retrieval quality strongly influences end-to-end MKG-RAG performance.
- Provides a foundation for diagnosing limitations in multimodal knowledge graph systems.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Retrieval-augmented generation (RAG) over knowledge graphs has emerged as a promising approach for grounding large language models, yet existing benchmarks largely overlook the challenges of retrieval in multimodal knowledge graph RAG (MKG-RAG). In practice, retrieval is a critical bottleneck: multimodal knowledge is heterogeneous, difficult to align across modalities, and often poorly served by retrievers designed for unstructured corpora. To address this gap, we introduce MKG-RAG-Bench, a cross-domain benchmark explicitly designed to evaluate retrieval in MKG-RAG. MKG-RAG-Bench is constructed from two multimodal knowledge graphs spanning general and medical domains, and includes carefully aligned question-answering datasets that support controlled evaluation of both retrieval and downstream generation. The benchmark is built using an LLM-based curation pipeline that filters low-utility knowledge, generates structurally grounded queries with exact supervision, and systematically covers diverse modality configurations. Through extensive experiments across representative retriever families and modality settings, we show that effective multimodal retrieval remains challenging yet crucial for end-to-end MKG-RAG performance, and that retrieval quality strongly determines generation outcomes. By isolating retrieval as a first-class evaluation target, MKG-RAG-Bench provides a principled foundation for diagnosing current limitations and advancing multimodal knowledge graph RAG systems.
| Comments: | Accepted by KDD'26 |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2606.26458 [cs.AI] |
| (or arXiv:2606.26458v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2606.26458 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xiaochen Wang [view email]
[v1]
Wed, 24 Jun 2026 23:38:42 UTC (476 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →The Verification Horizon: No Silver Bullet for Coding Agent Rewards
As coding agents evolve, verifying solutions becomes more challenging than generating them, necessitating a focus on scalable, faithful, and robust verification methods. The study reveals that no fixed reward function can sustain effectiveness as model capabilities advance, emphasizing the need for verification to evolve alongside solution generation.