CARGO-VL: Counterfactual Arbitration with Risk-Constrained Group Optimization for Vision-Language Models
Quick Answer
CARGO-VL introduces a novel framework for vision-language models that optimizes evidence states to improve conflict handling and answer reliability.
Quick Take
By integrating condition-wise correctness with adaptive risk control, it significantly enhances performance on benchmarks like CMC-Bench and Modality-Bias, outperforming traditional pointwise methods. This approach ensures better source trustworthiness and abstention when necessary.
Key Points
- CARGO-VL optimizes four evidence states: aligned, image-correct, text-correct, and both-wrong.
- The model improves conflict handling and unsupported-answer avoidance over pointwise baselines.
- A primal-dual controller balances unsafe answers against excessive deferral.
- XMC (eXtended Modal Conflict) is introduced as a training resource for conflict scenarios.
- CARGO-VL shows enhanced modality balance across multiple seeds in evaluations.
DeepSignal Analysis
What happened
CARGO-VL is a new framework designed for vision-language models that aims to improve conflict resolution and the reliability of answers by optimizing evidence states. It integrates condition-wise correctness with risk control, enhancing performance on specific benchmarks.
Key evidence
- CARGO-VL optimizes matched variants of evidence states, including aligned, image-correct, text-correct, and both-wrong, as a single bundle.
- The framework improves conflict handling and unsupported-answer avoidance on benchmarks like CMC-Bench and Modality-Bias compared to traditional methods.
- CARGO-VL includes a primal-dual controller that balances the risk of unsafe answers against excessive deferral, enhancing decision-making in uncertain scenarios.
Why it matters
The development of CARGO-VL addresses significant challenges in vision-language systems, particularly in identifying trustworthy sources and managing conflicting information. By improving the reliability of answers, this framework could enhance the practical applications of AI in areas requiring multimodal evidence, such as automated customer support and content generation.
What to watch
Paper Resources
📖 Reader Mode
~2 min readAbstract:Vision-language systems combine images with retrieved text, but these sources can disagree or jointly fail to support an answer. Reliable models must identify the trustworthy source and abstain when neither is adequate. Existing post-training objectives score instances independently and therefore do not enforce coherent behavior under counterfactual evidence changes. We introduce CARGO-VL, a group-relative framework that optimizes matched variants covering aligned, image-correct, text-correct, and both-wrong (A/V/T/N) evidence states as one bundle. Its objective couples condition-wise correctness with transition rewards for answer invariance, source equivariance, and answer-to-abstention switching, while a primal-dual controller balances unsafe answers against excessive deferral. We also contribute XMC (eXtended Modal Conflict), a four-condition conflict training resource, and evaluate transfer on CMC-Bench and Modality-Bias. Across multiple seeds, CARGO-VL improves conflict handling, unsupported-answer avoidance, and modality balance over pointwise baselines. Ablations identify complementary benefits from relational transition signals and adaptive risk control, supporting counterfactual consistency as a practical objective for reliable multimodal evidence arbitration.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2608.04509 [cs.AI] |
| (or arXiv:2608.04509v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2608.04509 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: De Jiang [view email]
[v1]
Wed, 5 Aug 2026 06:47:46 UTC (4,735 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →HOBA: Hierarchical On-Policy Bidding Agents for Adaptive Online Advertising
HOBA (Hierarchical On-policy Bidding Agents) is a novel hierarchical reinforcement learning framework that enhances online advertising bidding systems by improving adaptability and reducing hyperparameter tuning costs. It utilizes a for hyperparameter inference, a SARSA agent for expert model selection, and a dynamic expert pool for bid execution, achieving a +3.6% increase in target cost during large-scale deployment and outperforming state-of-the-art baselines on AuctionNet.