Better Starts, Better Ends: Bootstrapped Iterative Self-Reasoning Distillation for Compressed Reasoning
Quick Answer
The BIRD method enhances reasoning efficiency by improving model rollouts before training, achieving a MATH-500 accuracy increase from 86.2% to 92.0% while reducing response length from 3,099 to 1,115 tokens in Qwen3-8B.
Quick Take
This two-stage self-reasoning distillation addresses initialization bottlenecks in existing models, leading to better performance on benchmarks.
Key Points
- BIRD method samples concise solutions to enhance reasoning efficiency.
- Achieves MATH-500 accuracy improvement from 86.2% to 92.0% on Qwen3-8B.
- Reduces average response length from 3,099 to 1,115 tokens.
- Addresses initialization bottlenecks in existing self-distillation methods.
- Demonstrates prefix support as crucial for efficient reasoning distillation.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Large reasoning models often solve problems through long chain-of-thought (CoT) traces, yet much of this computation is spent on redundant derivations, repeated self-verification, and detours that do not improve the final answer. Existing on-policy self-distillation methods reduce this cost by matching a student model to a concise copy of itself on prefixes sampled from the student's own rollouts. We show that this objective has an initialization bottleneck. Since supervision is applied only to visited prefixes, training from a verbose base model places the KL loss on contexts that are often noisy, redundant, or already off track. In such regions, a concise teacher can provide only local corrections, while the student continues to explore trajectories that an efficient reasoner should avoid. In this paper, we propose BIRD(Bootstrapped Iterative Self-Reasoning Distillation), a two-stage self-reasoning distillation method that improves the rollout distribution before on-policy training. BIRD first samples concise solutions from the base model under a brevity instruction, keeps only answer-correct traces, and performs a lightweight prompt-switch SFT step. The traces are generated with the brevity instruction but learned under the original task prompt, turning instruction-induced conciseness into a default reasoning behavior. Starting from this warm model, BIRD then applies on-policy reverse-KL distillation with a concise self-teacher, now on cleaner and more informative prefixes. Across Qwen3 series models, BIRD achieves a stronger accuracy-efficiency trade-off than prompting and cold-start on-policy distillation on MATH-500 and AIME benchmarks. On Qwen3-8B, it improves MATH-500 accuracy from 86.2% to 92.0% while reducing the average response length from 3,099 to 1,115 tokens. These results highlight prefix support as a central factor in efficient reasoning distillation.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2607.15736 [cs.CL] |
| (or arXiv:2607.15736v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.15736 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Leichao Dong [view email]
[v1]
Fri, 17 Jul 2026 08:15:35 UTC (865 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.