Not All LLM Reasoning is Visible in the Chain-of-Thought
Quick Answer
This paper shows that Frontier language models, like Claude Opus 4.5 and Qwen3-235B, utilize semantically irrelevant filler tokens to enhance performance on reasoning tasks, achieving accuracy gains of up to 13%.
Quick Take
This 'invisible reasoning' occurs without interpretable traces in output tokens, raising concerns for AI safety.
Key Points
- 13 frontier models evaluated across three synthetic reasoning tasks.
- Filler tokens improved accuracy by up to 13 percentage points.
- Claude Opus 4.5 satisfied hidden modular arithmetic constraints using filler tokens.
- Reinforcement learning influenced filler token preferences but lacked test-time benefits.
- Invisible reasoning poses challenges for AI interpretability and safety.
DeepSignal Analysis
What happened
Recent research indicates that advanced language models, such as Claude Opus 4.5 and Qwen3-235B, utilize semantically irrelevant filler tokens to enhance their reasoning capabilities. This approach has led to accuracy improvements of up to 13 percentage points in specific tasks, highlighting a form of 'invisible reasoning' that is not reflected in the model's output.
Key evidence
- Claude Opus 4.5 and Qwen3-235B have shown significant performance gains by using filler tokens, achieving accuracy improvements of up to 13 percentage points.
- The study evaluated 13 frontier language models across three tasks, revealing that many benefited from the inclusion of filler tokens.
- Filler tokens allowed Claude Opus 4.5 to meet a hidden modular arithmetic constraint without affecting its primary task accuracy.
Why it matters
The findings raise important questions regarding AI safety, as the reasoning processes of these models may not be fully transparent. The use of filler tokens to achieve better performance without interpretable traces could lead to unforeseen consequences in critical applications, where understanding model behavior is essential for trust and reliability.
Paper Resources
Source Excerpt
A key question for AI safety is whether a language model expresses all of its reasoning in its output tokens. We demonstrate a concrete failure mode where frontier models exhibit invisible reasoning by leveraging semantically irrelevant filler tokens to improve performance on synthetic reasoning tasks. We evaluate 13 frontier language models across three tasks and find that many models benefit significantly from filler tokens, with accuracy improvements of up to 13 percentage points. The benefit
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.