A Cost-Aware, Paired Protocol for Auditing Dynamic Tool Synthesis in Agentic Video Question Answering
Quick Answer
The proposed cost-aware, paired protocol audits tool-augmented VideoQA systems, revealing that Dynamic-SAGE improves accuracy by 7.5 points while increasing inference costs by 26%.
Quick Take
This method evaluates both accuracy and cost jointly, providing insights into efficiency gains and regressions across different question types.
Key Points
- Dynamic-SAGE improves accuracy by 7.5 points (p < 0.001) over the SAGE baseline.
- Inference costs rise by 26% with a 34% increase in token usage.
- Visible tool calls and reasoning turns are reduced by approximately 28%.
- Performance gains are largest on visual and open-ended questions.
- The protocol uses McNemar's test for significance and paired bootstrap confidence intervals.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Agentic Video Question Answering (VideoQA) systems invoke tools during inference, but their tool libraries are fixed, so recurring procedures are rebuilt from primitives on every question. Synthesizing composite tools could remove this overhead, but whether such expansion helps is hard to assess: final-answer accuracy, the standard metric, ignores inference effort, so it cannot reveal how a system shifts cost. We propose a cost-aware, paired protocol for auditing tool-augmented video agents. The protocol pairs two complete systems on the same input for each question and reports their net difference across accuracy and cost jointly. For each question, it sorts the paired outcome into one of six groups defined by joint correctness and by the change in visible tool calls, separating accuracy-preserving efficiency gains from harmful regressions. Significance is reported with McNemar's test and paired bootstrap confidence intervals. We instantiate the protocol on Dynamic-SAGE, an agentic VideoQA framework that synthesizes, validates, and persistently registers executable composite tools for reuse on unseen questions, and evaluate it against the SAGE baseline on SAGE-Bench. The audit reveals a multi-axis profile that a scalar accuracy comparison would miss: Dynamic-SAGE improves accuracy by 7.5 points (p < 0.001) and reduces reasoning turns and visible tool calls by roughly 28%, while shifting rather than reducing inference cost, as token usage rises 34% and cost 26%. Gains are largest on visual and open-ended questions and neutral on verbal and multimodal ones, and residual failures concentrate on hard, open-ended questions where the pipeline does the most work. By measuring accuracy and cost jointly, the protocol shows where the pipeline-level difference is reliable and where it is not. The code is available at this https URL.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2607.01469 [cs.CV] |
| (or arXiv:2607.01469v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2607.01469 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Rasul Khanbayov [view email]
[v1]
Wed, 1 Jul 2026 21:01:57 UTC (6,351 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
ProMoE-FL introduces a Prototype-conditioned Mixture-of-Experts framework for multimodal federated learning, effectively addressing missing modalities. It outperforms existing methods on four chest X-ray datasets, demonstrating superior feature synthesis capabilities in both homogeneous and heterogeneous settings.