Task-Specific Multimodal Question Answering Agents via Confidence Calibration and Incremental Reasoning for QANTA 2026
Quick Answer
This paper shows that The submission for QANTA 2026 introduces a two-agent architecture leveraging GPT-4 models for multimodal question answering, achieving a top leaderboard score of 0.402.
Quick Take
The submission for QANTA 2026 introduces a two-agent architecture leveraging GPT-4 models for multimodal question answering, achieving a top leaderboard score of 0.402. The Tossup agent employs confidence calibration and numeric reasoning, while the Bonus agent integrates multimodal evidence for precise answer selection, demonstrating effective performance under efficiency constraints.
Key Points
- Developed a task-specific two-agent architecture for QANTA 2026 challenge.
- Tossup agent uses GPT-4.1-mini with confidence calibration and numeric reasoning.
- Bonus agent employs GPT-4.1 for structured reasoning and multimodal integration.
- Achieved highest leaderboard score of 0.402 in resource-constrained benchmarks.
- Demonstrates effective lightweight reasoning strategies for multimodal question answering.
Paper Resources
📖 Reader Mode
~2 min readAbstract:We present our submission to the QANTA 2026 shared challenge at the ICML 2026 Workshop on Efficient Multimodal Question Answering (EMM-QA). Quanta evaluates multimodal quizbowl systems that answer pyramid-style questions from incrementally revealed text and accompanying images while operating under realistic efficiency constraints. The challenge consists of two distinct tasks: Tossup questions, which require deciding when to answer under uncertainty, and Bonus questions, which emphasize accurate answer selection and human adoption. To address these differing objectives, we develop a task-specific two-agent architecture. Our Tossup agent utilizes a GPT-4o-mini-class model (referred to as GPT-4.1-mini in the competition logs) with confidence-calibrated answering and a domain-specific numeric reasoning policy that reduces overconfident predictions from isolated quantitative clues. Our Bonus agent uses GPT-4o-class model (referred to as GPT-4.1) with leadin-aware reasoning, structured relational reasoning, and multimodal evidence integration to improve exact answer selection. Rather than relying on a retrieval pipeline or model ensembles, our approach emphasizes efficient reasoning policies and confidence calibration within a hosted-only environment. Our system achieved the highest overall leaderboard score of 0.402, including a Tossup score of 0.238 and a Bonus Effect score of 0.164. The results demonstrate that lightweight, task-specific reasoning strategies can provide strong performance on resource-constrained multimodal question answering benchmarks.
| Comments: | 10 pages, 1 figure. Accepted at the EMM-QA 2026 Workshop, ICML 2026 (Non-Archival). Rank #1 overall system in the QANTA 2026 Challenge |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.09623 [cs.CL] |
| (or arXiv:2607.09623v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.09623 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Nirjhar Das [view email]
[v1]
Fri, 10 Jul 2026 17:22:49 UTC (171 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Letting the Data Speak: Extracting Keywords from Crowdsourced Collections with AI
The study evaluates three NLP approaches—Named Entity Recognition, Keyword Extraction, and Topic Modelling—using the Their Finest Hour Online Archive to automate keyword extraction from crowdsourced WWII collections. Findings suggest that while NLP methods show promise, no single approach is sufficient, and ethical considerations in automated keyword extraction are crucial for responsible stewardship.