Can Generalist Agents Automate Data Curation?
Quick Answer
This paper shows that Generalist coding agents can automate data curation loops, achieving strong data-selection baselines in vision-language tasks with *Curation-Bench*.
Quick Take
However, they primarily refine existing policies rather than innovate, necessitating scaffolded methods for effective exploration. The scaffolded agents autonomously developed superior data-selection policies at a fraction of the data budget.
Key Points
- Agents achieved strong data-selection baselines within ten iterations using *Curation-Bench*.
- Execution-research gap revealed agents mainly tuning local policy variants.
- Scaffolded methods shifted agents towards method-guided exploration.
- Autonomous composition of data-selection policy outperformed baselines at one-tenth the data budget.
- Code and benchmark are available as open-source.
Paper Resources
Source Excerpt
From the original publisher, up to about 700 charactersarXiv:2606. 04261v1 Announce Type: new Abstract: Curating training data is among the most consequential yet labor-intensive parts of modern AI development: practitioners iteratively propose, implement, evaluate, and revise data policies against noisy benchmark feedback. We ask whether generalist coding agents can automate this data-curation loop.
We introduce *Curation-Bench*, an agent-centric benchmark that fixes the model, training recipe, and evaluation suite while giving agents command-line access to inspect data, implement policies, submit them to a fixed training/evaluation pipeline, and revise. …
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →Automatic Ordinary Differential Equations Discovery For Biological Systems Using Powered Agentic System
The MEDA system utilizes large language models and symbolic regression to autonomously discover ordinary differential equations for biological systems, achieving strong structural recovery and biologically plausible models. It outperforms existing methods by integrating domain knowledge and mechanistic constraints, demonstrating effective retrieval and extrapolation capabilities.