Beyond Binary Verifiable Rewards: Behavioral Self-Rewarding for SWE Agents
Quick Answer
The paper introduces Behavioral Self-Rewarding (BSR) for reinforcement learning in software engineering, enhancing performance by allowing agents to self-evaluate debugging quality without a separate reward model.
Quick Take
Experiments on configurations show consistent improvements in Qwen3-14B and Qwen3-32B, leveraging comparative judgments from existing trajectories.
Key Points
- BSR aggregates pairwise preferences into task-local scores for improved reward signals.
- The method preserves pass/fail ordering defined by the verifier while introducing reward contrast.
- Experiments demonstrate consistent performance improvements in Qwen3-14B and Qwen3-32B models.
- No new environment interactions are required for additional supervision.
- This approach enhances the learning signal for software engineering agents.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Reinforcement learning with verifiable rewards (RLVR) provides reliable supervision for software engineering agents through executable tests, but binary rewards fail to distinguish trajectories with substantially different debugging quality. We investigate whether the current policy can judge its own trajectories and use these judgments to improve task performance without a separately trained reward model. To this end, we introduce Behavioral Self-Rewarding (BSR), a reward-refinement method that reuses the policy to compare observable debugging behavior among trajectories from the same task with the same verifier outcome. BSR aggregates pairwise preferences into task-local scores and converts them into bounded auxiliary rewards, introducing reward contrast while preserving the verifier-defined pass/fail ordering. This provides additional supervision from already-collected trajectories without requiring new environment interactions. Experiments on SWE-bench Verified and SWE-bench-Live show consistent improvements across the evaluated Qwen3-14B and Qwen3-32B configurations, supporting using the policy's own comparative judgments as a complementary learning signal for training software engineering agents.
| Subjects: | Artificial Intelligence (cs.AI); Software Engineering (cs.SE) |
| Cite as: | arXiv:2610.12598 [cs.AI] |
| (or arXiv:2610.12598v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.12598 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yiran Wu [view email]
[v1]
Thu, 8 Oct 2026 18:21:10 UTC (591 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →VEPA: Verified Evolving Formal Policy for Lifelong Agent Guardrails
VEPA introduces a framework for verified evolving formal policies for LLM agents, enhancing safety by adapting guardrails based on deployment feedback. It outperforms existing methods across benchmarks like ASSEBench-Safety and ASSEBench-Security, ensuring deterministic policy enforcement even under challenging conditions.