AI coding agents find the right file but miss the exact lines that matter, study shows
Quick Answer
AI coding agents like Claude Code and Codex can locate the correct file but often overlook critical lines necessary for code repair.
Quick Take
The SWE-Explore benchmark reveals that without sufficient context, even the best solutions can fail, highlighting a significant gap in current AI capabilities.
Key Points
- Claude Code and Codex are effective at locating files but miss critical lines.
- The SWE-Explore benchmark tests code search separately from repair tasks.
- Insufficient context leads to failures in AI-generated code fixes.
- The study emphasizes the limitations of current AI coding agents.
Source Excerpt
AI coding agents like Claude Code or Codex reliably find the right file but miss most of the critical lines within it. The new SWE-Explore benchmark is the first to test code search separately from the actual repair, and it shows that without enough context, even the best fix will fail.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from The Decoder
See more →
An AI model programmed nonstop for 19 days on a single MirrorCode task that cost $2,600 to run
Epoch AI's MirrorCode benchmark reveals Claude Opus 4.7 as the leader with a 56% solve rate, reconstructing a 16,000-line toolkit in 14 hours. Despite this, all models tested struggle with the most complex tasks, highlighting limitations in current AI capabilities. The single task consumed $2,600 over 19 days, raising questions about cost-effectiveness in AI development.
