Heuresis: Search Strategies for Autonomous AI Research Agents Across Quality, Diversity and Novelty
Quick Answer
Heuresis is a new framework for autonomous AI research agents that enhances exploration of quality, diversity, and novelty in machine learning.
Quick Take
Heuresis is a new framework for autonomous AI research agents that enhances exploration of quality, diversity, and novelty in machine learning. It implements six search strategies and evaluates them across 3,222 runs, revealing that truly novel ideas are rare and often do not outperform established methods. The findings highlight the need for improved strategies to bridge the gap in quality-novelty exploration.
Key Points
- Heuresis abstracts the research pipeline into general and composable primitives.
- Implemented six search strategies, including MAP-Elites and Curiosity.
- Only one novel idea ranked in the top-10 for quality across all runs.
- Agents exhibited reward-hacking techniques in 40 confirmed cases.
- Current strategies do not expand the quality-novelty frontier.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Autonomous AI Research promises to accelerate the scientific progress of machine learning. To realise this goal, current Large Language Model (LLM)-based agents need to go beyond just writing code, to mastering the exploration of simultaneously performant, diverse and novel ideas. To this end, we introduce Heuresis, a framework that abstracts the research pipeline into a set of general and composable primitives, enabling open-ended scientific exploration in machine learning research. We implement six search strategies: a greedy baseline, two archive-based (MAP-Elites, Go-Explore), one evolutionary (Islands), and two divergent (Curiosity, Omni), and evaluate them across three axes (Quality, Diversity, and Novelty) on three domains (LLM Pretraining, On-Policy RL, and Model Unlearning), totalling 3,222 scored runs. We find that completely novel ideas are rare. No idea across our scored runs is rated as "Original", and only a few achieve only "Minor Similarity" to prior work. Moreover, novel ideas never approach the highest-performing known-recipe scores. Across all six strategies and three domains, only one such idea lands in the top-10 by quality. We also observed agents resorting to a variety of reward-hacking techniques during execution (40 confirmed fabrications across 1,628 scored runs), and detecting them was necessary to keep the search faithful to the task. Our results show that while current search and Quality-Diversity strategies enable us to steer where the generated ideas land on the quality, diversity, and novelty axes, they do not expand the quality-novelty frontier. Bridging this gap is the open challenge towards the ultimate goal of perpetual, autonomous scientific progress. Code is available at this http URL.
| Comments: | 14 pages main text, 82 pages total including appendix; 38 figures, 4 tables |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2606.25198 [cs.AI] |
| (or arXiv:2606.25198v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2606.25198 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Antonis Antoniades [view email]
[v1]
Tue, 23 Jun 2026 21:44:08 UTC (6,269 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →The Verification Horizon: No Silver Bullet for Coding Agent Rewards
As coding agents evolve, verifying solutions becomes more challenging than generating them, necessitating a focus on scalable, faithful, and robust verification methods. The study reveals that no fixed reward function can sustain effectiveness as model capabilities advance, emphasizing the need for verification to evolve alongside solution generation.