Closing the Reflection Gap: A Free Calibration Bonus for Agentic RL
Quick Answer
The proposed RefGRPO method enhances agentic reinforcement learning by introducing a calibration bonus that improves reflection accuracy and task performance.
Quick Take
It reduces the underconfidence rate from 44.4% to 7.7% and increases task accuracy from 75.1% to 76.5% across five text-to-SQL benchmarks, enabling better self-improvement and selective prediction.
Key Points
- RefGRPO augments standard RL with a free calibration bonus for improved performance.
- Underconfidence rate decreased from 44.4% to 7.7% with RefGRPO implementation.
- Task accuracy improved from 75.1% to 76.5% across five text-to-SQL benchmarks.
- Agents can use reflections as pseudo-rewards for self-improvement without supervision.
- Dynamic scheduling of calibration coefficient enhances agent decision-making.
Paper Resources
Source Excerpt
arXiv:2606. 14211v1 Announce Type: new Abstract: are increasingly deployed as agents that interact with external environments and observe feedback such as execution results, error messages, and tool outputs. A well-functioning agent should be able to leverage this feedback to accurately assess its own performance.
Yet we find a persistent reflection gap: LLM agents tend to mis-assess their own outputs after observing concrete environment feedback -- even for questions they correctly answered -- and standard RL barely helps due to a credit-assignment mismatch. …
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →HOBA: Hierarchical On-Policy Bidding Agents for Adaptive Online Advertising
HOBA (Hierarchical On-policy Bidding Agents) is a novel hierarchical reinforcement learning framework that enhances online advertising bidding systems by improving adaptability and reducing hyperparameter tuning costs. It utilizes a for hyperparameter inference, a SARSA agent for expert model selection, and a dynamic expert pool for bid execution, achieving a +3.6% increase in target cost during large-scale deployment and outperforming state-of-the-art baselines on AuctionNet.