Recursive Self-Evolving Agents via Held-Out Selection
Quick Answer
This paper shows that The Recursive Self-Evolving Agent (RSEA) outperforms existing methods like ReAct on the ALFWorld benchmark, achieving 69.3% accuracy, and 79.4% with retries.
Quick Take
The Recursive Self-Evolving Agent (RSEA) outperforms existing methods like ReAct on the ALFWorld benchmark, achieving 69.3% accuracy, and 79.4% with retries. RSEA's strict held-out selection ensures it never underperforms compared to the base agent, while unguarded context evolution proves to be high-variance and unsafe across tasks. Overall, no single artifact consistently excels across benchmarks.
Key Points
- RSEA achieves 69.3% accuracy on ALFWorld, surpassing ReAct's 64.6%.
- Best overall performance of 79.4% on ALFWorld with retries.
- Dynamic Cheatsheet shows high variance, collapsing on WebShop.
- RSEA's strict selection prevents significant underperformance compared to base agents.
- No single artifact consistently wins across multiple benchmarks.
Paper Resources
📖 Reader Mode
~2 min readAbstract:LLM agents are increasingly improved without weight updates by evolving a natural-language artifact, such as reflections, workflows, playbooks, cheatsheets, or optimized prompts, that conditions a frozen policy. Such methods are typically reported as wins on the single benchmark where they help. We study them apples-to-apples and surface a sharper picture. We introduce RSEA, a Recursive Self-Evolving Agent that carries a compact three-layer natural-language state: an imperative strategy, reusable skills, and a procedural playbook. Across generations, RSEA rewrites all three layers from its own trajectories and commits a candidate only if it does not regress on a disjoint held-out split, using a strict keep-better gate.
Across four diverse benchmarks, ALFWorld, GAIA, (\tau)-bench, and WebShop, and six faithful baselines, ReAct, Reflexion, GEPA, AWM, ACE, and Dynamic Cheatsheet, all evaluated on one shared local backbone, we find three main results. First, no artifact universally wins. RSEA is the strongest single-pass method on ALFWorld, reaching 69.3% compared with 64.6% for ReAct (McNemar (p=0.015)), and reaches 79.4% with retry, the best overall result. However, concrete-workflow induction, represented by AWM, is best on the strong-backbone tool-use tasks. Second, unguarded context evolution is high-variance and unsafe. Dynamic Cheatsheet, which curates context online without a held-out gate, is near-best on ALFWorld at 70.7%, yet collapses on WebShop, with a score of 0.14 compared with 0.43 for ReAct. Third, RSEA's strict held-out selection is what makes recursive self-evolution monotone-safe: it never significantly underperforms the base agent on any benchmark and falls back to vanilla ReAct when evolved context would hurt.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2606.28374 [cs.AI] |
| (or arXiv:2606.28374v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2606.28374 arXiv-issued DOI via DataCite |
Submission history
From: Michael Nguyen [view email]
[v1]
Wed, 17 Jun 2026 14:53:36 UTC (257 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →The Verification Horizon: No Silver Bullet for Coding Agent Rewards
As coding agents evolve, verifying solutions becomes more challenging than generating them, necessitating a focus on scalable, faithful, and robust verification methods. The study reveals that no fixed reward function can sustain effectiveness as model capabilities advance, emphasizing the need for verification to evolve alongside solution generation.