CacheRL:Multi-Turn Tool-Calling Agents via Cached Rollouts and Hybrid Reward
Quick Answer
This paper shows that CacheRL trains small agent models achieving 92% accuracy on multi-step tool-calling tasks, nearing GPT-5's 94% while using 100x less compute.
Quick Take
Key innovations include a hybrid thinking trajectory pipeline, a three-tier fuzzy cache, and cache-aware rewards, enhancing performance significantly against leading models.
Key Points
- Achieves 92% accuracy on tool-calling tasks, close to GPT-5's 94%.
- Uses 100 times less compute than larger models for training.
- Introduces hybrid thinking trajectories for enhanced learning.
- Implements a three-tier fuzzy cache to eliminate live execution costs.
- Cache-aware rewards improve performance by 17% in benchmarks.
Paper Resources
Source Excerpt
arXiv:2606. 14179v1 Announce Type: new Abstract: We present CacheRL, a system for training small agent foundation models that achieves 92 percent process accuracy on multi-step tool-calling tasks, approaching GPT-5's 94 percent while requiring 100 times less compute.
Our approach addresses three challenges in practical agent training: transferring tool-calling knowledge from large models at scale, enabling reinforcement learning without costly live tool execution, and learning robustly from noisy cached environments. CacheRL introduces three key innovations. …
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.