Do Coding Agents Need Executable World Models, Simplification, and Verification to Solve ARC-AGI-3?
Quick Answer
The study evaluates four Codex-based agents on ARC-AGI-3, revealing that while all variants improve with stronger models and reasoning effort, the textual variant outperformed the flexible executable model in certain settings.
Quick Take
The complete verification treatment consistently ranked highest but required more resources, achieving 99% RHAE in follow-ups with gpt-5.6-sol.
Key Points
- Four Codex-based agents tested: textual baseline, executable model, and verification variants.
- Textual variant outperformed flexible executable model in gpt-5.5 settings.
- Complete verification treatment ranked first in all settings, using more resources.
- Simplification improved performance in three out of four model-effort settings.
- gpt-5.6-sol variant solved all public games with 99% RHAE, using fewer actions than humans.
Paper Resources
Source Excerpt
Our previous -3 agent bundled executable world modeling, scheduled simplification, and exact replay verification, leaving unclear which idea accounted for its performance. We address this attribution question with four nested Codex-based agents: a textual baseline; a flexible-interface executable world model without replay verification; the same executable model with scheduled simplification; and a fixed-interface verification treatment that retains simplification and requires exact repro
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →Automatic Ordinary Differential Equations Discovery For Biological Systems Using Powered Agentic System
The MEDA system utilizes large language models and symbolic regression to autonomously discover ordinary differential equations for biological systems, achieving strong structural recovery and biologically plausible models. It outperforms existing methods by integrating domain knowledge and mechanistic constraints, demonstrating effective retrieval and extrapolation capabilities.