SENTINEL: Failure-Driven Reinforcement Learning for Training Tool-Using Language Model Agents
Quick Answer
SENTINEL is a failure-driven reinforcement learning framework that enhances tool-using language model agents by turning rollout failures into targeted training tasks.
Quick Take
Tested on Tau2-Bench Retail with Qwen3-4B-Thinking-2507, it improved Pass^1 scores from 66.4 to 74.9, outperforming traditional RL methods on synthetic tasks.
Key Points
- SENTINEL uses a Controller-Proposer-Solver loop to analyze and address model failures.
- The framework generates tasks that specifically target recurring error patterns in agent performance.
- SENTINEL achieved a Pass^1 score improvement of 8.5 points on Tau2-Bench Retail.
- It outperformed traditional reinforcement learning on general synthetic tasks across Pass^k metrics.
- The approach demonstrates a scalable method for enhancing tool-using language model agents.
Paper Resources
Source Excerpt
Language model agents are increasingly effective in solving realistic tasks through multi-turn . However, training reliable tool-using agents remains challenging in practice. While reinforcement learning provides an on-policy paradigm for improving agents from their own environment interactions, its effectiveness depends heavily on the training task distribution. When tasks are fixed before training, the task distribution can become increasingly mismatched with the policy's evolving capa
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.