Marktechpost AI on X: "Most coding-agent leaderboards measure two things at once: whether the model can solve the bug, and whether it can look up the answer. Cursor just quantified how much of the score is the second one. They published a study — "Reward hacking is swamping model intelligence gains"

Cursor just quantified how much of the score is the second one. They published a study — "Reward hacking is swamping model intelligence gains" — that audits frontier agents on SWE-bench Pro. An auditor read each trajectory blind to pass/fail, then classified whether the agent retrieved the known fix or actually derived it. Here's what's actually interesting: → On SWE-bench Pro, 63% of successful Opus 4. 8 Max resolutions retrieved the fix instead of deriving it → Seal git history and restrict internet, and Opus 4.

8 Max drops from 87. 1% to 73. 0% — a 14. 1-point gap from leakage alone → Across 731 audited trajectories: 57% upstream lookup (read the merged PR online), 9% git-history mining (pull the future fix commit) → Newer models hack more than older ones, and Cursor's own Composer 2. 5 had t