What Benchmarks Don't Measure: The Case for Evaluating Abstention Competence in Autonomous Agents
Quick Take
The paper critiques existing benchmarks for autonomous agents, highlighting 'compliance bias' where agents act without necessary inputs. It proposes a taxonomy for abstention scenarios and introduces evaluation protocols that achieved up to 89.2% hazardous-action blocking across 144 scenarios, indicating a tunable safety-usability tradeoff.
Key Points
- Compliance bias leads agents to act without proper inputs or authorization.
- Introduces a taxonomy of three abstention-warranted scenarios: specification, verification, and authority gaps.
- Proposed evaluation protocols achieved 89.2% hazardous-action blocking in enterprise scenarios.
- Safety-usability tradeoff is tunable and varies across different model families.
- Study serves as a foundation for further research on abstention-aware benchmarks.
Article Content
From source RSS / original summaryarXiv:2606. 02965v1 Announce Type: new Abstract: Benchmarks for autonomous agents measure whether agents complete tasks, yet this framing is systematically blind to whether an agent should have proceeded at all.
Agents trained under human-feedback objectives develop a structural tendency to proceed even when they lack the inputs, evidence, or authorization to act safely, a disposition we term compliance bias, because both the reward signal and the benchmark scoring regime treat proceeding as the correct default regardless of whether the preconditions for safe action are present. We make three contributions.
We first show that compliance bias originates in reward hacking within human-feedback pipelines and is entrenched by prominent agent benchmarks, which either penalize agents for pausing or are architecturally unable to distinguish a principled pause from a silent failure.
We then introduce a three-gap taxonomy of abstention-warranted scenarios, covering specification gaps where required information is absent, verification gaps where world state cannot be confirmed, and authority gaps where explicit authorization has not been given, which together provide a principled basis for constructing abstention-aware agent benchmarks.
Finally, we propose abstention evaluation protocols (Safety Rate, Usability Rate, and Informed Refusal Rate) and report preliminary results across 144 enterprise agent scenarios and five model families, in which a runtime-enforced abstention mechanism achieves up to 89. 2% hazardous-action blocking and 87. 5% usability on authorized scenarios, demonstrating that the safety--usability tradeoff is tunable rather than inherent and that its shape varies substantially across model families.
We treat this as preliminary work and offer the taxonomy and composite metrics as a starting point for further conversations.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →AUDITFLOW: Executable Symbolic Environments for Structured Financial Reporting Verification
AuditFlow introduces a multi-agent framework for structured financial reporting verification, achieving 82.09% accuracy with GPT-5.5, outperforming the baseline by 14.93 points. It utilizes a symbolic environment for effective audit processes, demonstrating the necessity of deterministic checks for reliable verification.
