How Much Thinking is Enough? Quantifying and Understanding Redundancy in LLM Reasoning
Quick Take
This study quantifies reasoning redundancy in large language models (LLMs) like GPT-3, revealing that 61-93% of reasoning steps can be truncated without loss of accuracy. The findings indicate that overthinking is a structural issue in LLMs, not a model-specific flaw, impacting efficiency across various benchmarks.
Key Points
- Step-level redundancy in LLMs is consistently high, between 61% and 93%.
- Median critical prefix is often just a single segmented step in most conditions.
- Redundancy decreases with problem difficulty but remains substantial even on Level-5 problems.
- Overthinking is linked to length-agnostic outcome rewards, not specific to individual models.
- Findings apply across four reasoning models and two mathematical benchmarks.
Article Content
From source RSS / original summaryarXiv:2605. 23926v1 Announce Type: new Abstract: Reasoning-capable large language models solve hard problems by emitting long chains of thought, paying heavily in latency, GPU time, and energy. Casual inspection of their traces reveals extensive reformulation, verification, and circular self-reflection, yet how much of this deliberation is actually necessary has never been measured at scale or explained from first principles. This paper closes both gaps.
We formalise reasoning redundancy directly in terms of the reasoning model itself: the redundancy of a correct trace is the largest fraction of its trailing segmented steps that can be truncated while $\pi$, forced to terminate thinking and emit a final answer, still produces the correct answer.
A large-scale quantification across four frontier reasoning models and two mathematical benchmarks shows that step-level redundancy is consistently high -- between 61% and 93% across the 8 (model, benchmark) conditions we study, with the median critical prefix equal to a single segmented step in six of the eight conditions -- that the finding is robust to the choice of judge family, and that although $\rho$ decreases with problem difficulty on MATH-500, all four models remain substantially redundant ($\rho \in [46\%, 85\%]$) even on the hardest Level-5 problems.
We then prove that this redundancy is a structural consequence of length-agnostic outcome rewards, not a model-specific artefact: under any such reward, no finite expected stopping time is optimal. The result holds regardless of RL algorithm, base model, data distribution, or whether the policy is obtained via RL or distillation; over-thinking is therefore not a bug to be patched in individual models but a structural property of how current reasoning models are trained. Code: https://github.
com/zhiyuanZhai20/how-much-thinking-is-enough
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →The Importance of Out-of-Band Metadata for Safe Autonomous Agents: The Redpanda Agentic Data Plane
The Redpanda Agentic Data Plane (ADP) introduces out-of-band metadata channels to enhance the safety of autonomous AI agents, ensuring secure data access and tamper-proof audit trails. This architecture mitigates risks associated with unpredictable AI behavior by enforcing governance throughout the agent lifecycle, demonstrated in a multi-agent trading system with strict data scoping and approval thresholds.