HAVEN: Hierarchically Aligned Multimodal Benchmark for Unified Video Understanding
Quick Take
HAVEN introduces a comprehensive benchmark for unified video understanding with hierarchical multimodal alignment.
Key Points
- Addresses gaps in existing multimodal evaluation benchmarks.
- Offers a unified dataset architecture for video and text.
- Provides a rigorous testbed for future multimodal research.
📖 Reader Mode
~2 min readAbstract:While Multimodal Large Language Models (MLLMs) exhibit strong performance on standard video tasks, their ability to faithfully summarize and reason over complex narratives remains poorly evaluated. Existing summarization benchmarks fragment supervision across isolated granularities, such as keyframes, key shots, or disjointed text summaries, failing to capture the inherently hierarchical structure of cross-modal alignment. To address this critical gap, we introduce HAVEN, a hierarchically aligned multimodal benchmark for unified video understanding. HAVEN pioneers a fully granular (frame, shot, and video levels) and fully multimodal (video and text) dataset architecture, complete with explicit, continuous alignment between modalities. Built upon this unified annotation paradigm, we propose a comprehensive evaluation suite spanning summarization, temporal reasoning, multimodal grounding, and saliency ranking. Extensive benchmarking of state-of-the-art MLLMs exposes a persistent gap between surface-level textual fluency and grounded multimodal understanding. Ultimately, HAVEN advances the evaluation of multimodal systems beyond traditional QA formats, offering a rigorous, standardized testbed to drive future research in interpretable, hierarchical video understanding. We publicly release the dataset, benchmark suite, and evaluation protocols.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2605.19223 [cs.CV] |
| (or arXiv:2605.19223v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2605.19223 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Haopeng Zhang [view email]
[v1]
Tue, 19 May 2026 00:48:14 UTC (9,750 KB)
— Originally published at arxiv.org
More from arXiv cs.CV
See more →arXiv cs.CV·Jinhao Jing, Zheng Ma, Jinwei Liang, Qiannian Zhao, Shawn Chen, Jing Yang, Por Lip Yee, Prayag Tiwari, Jingjing Bai, Benyou Wang, Lewei Lu, Zhan Su
1d ago FeaturedOriginal
GeoSym127K: Scalable Symbolically-verifiable Synthesis for Multimodal Geometric Reasoning
AI Summary
GeoSym127K introduces a scalable neuro-symbolic framework for enhanced geometric reasoning in multimodal models.