CRAFT: Critic-Refined Adaptive Key-Frame Targeting for Multimodal Video Question Answering
arXiv cs.CV·Mahesh Bhosale, Abdul Wasi, Vishvesh Trivedi, Pengyu Yan, Akhil Gorugantu, David Doermann
17h agoQuick Take
CRAFT enhances multimodal video question answering with adaptive key-frame targeting and a hybrid critic loop.
Key Points
- Integrates dynamic keyframe selection and multilingual ASR.
- Achieves top performance on MAGMaR 2026 benchmarks.
- Code available at GitHub for public access.
📖 Reader Mode
~2 min readAbstract:Grounded multi-video question answering over real-world news events requires systems to surface query-relevant evidence across heterogeneous video archives while attributing every claim to its supporting source. We introduce CRAFT (Critic-Refined Adaptive Key-Frame Targeting), a query-conditioned pipeline that combines dynamic keyframe selection, per-video ASR with multilingual fallback, and a hybrid critic loop to iteratively verify and repair claims before consolidation. The pipeline integrates UNLI temporal entailment, DeBERTa-v3 cross-claim screening, and a Llama-3.2-3B adjudicator, with a final citation-merging stage that emits each fact once with all supporting source identifiers. On MAGMaR 2026, CRAFT achieves the best overall average (0.739), reference recall (0.810), and citation F1 (0.635). We further evaluate on a MAGMaR-style conversion of WikiVideo with 52 non-overlapping event queries, where CRAFT also performs strongly (0.823 Avg), showing that its claim-centric evidence aggregation generalizes beyond MAGMaR. Ablations show that atomic claims, ASR, and the critic loop drive the main gains over the vanilla query-conditioned baseline. Code and implementation details are publicly available at this https URL.
| Comments: | Accepted at ACL 2026 Multimodal Augmented Generation via MultimodAl Retrieval Workshop |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.19075 [cs.CV] |
| (or arXiv:2605.19075v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2605.19075 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Vishvesh Trivedi [view email]
[v1]
Mon, 18 May 2026 20:01:05 UTC (137 KB)
— Originally published at arxiv.org
More from arXiv cs.CV
See more →arXiv cs.CV·Jinhao Jing, Zheng Ma, Jinwei Liang, Qiannian Zhao, Shawn Chen, Jing Yang, Por Lip Yee, Prayag Tiwari, Jingjing Bai, Benyou Wang, Lewei Lu, Zhan Su
1d ago FeaturedOriginal
GeoSym127K: Scalable Symbolically-verifiable Synthesis for Multimodal Geometric Reasoning
AI Summary
GeoSym127K introduces a scalable neuro-symbolic framework for enhanced geometric reasoning in multimodal models.