Self-Verified Distillation: Your Language Model Is Secretly Its Own Synthetic Data Pipeline
Quick Answer
This paper shows that Self-Verified Distillation allows large language models like Qwen3 to self-improve using unlabeled prompts, achieving significant performance gains: +16.7 in math, +11.1 in science, and +8.3 in coding.
Quick Take
This method enhances reasoning without external feedback, requiring only a single inference call during testing.
Key Points
- Self-Verified Distillation uses unlabeled seed questions for model self-improvement.
- Qwen3-4B model shows +16.7 points in math benchmarks after applying the method.
- The approach filters generated solutions through a three-stage validation process.
- Performance improvements extend to smaller models (0.6B and 8B).
- Only one inference call is needed at test time for enhanced performance.
Paper Resources
Article Content
From source RSS / original summaryarXiv:2605. 26132v1 Announce Type: new Abstract: Can post-trained (LLMs) further improve themselves using only unlabeled prompts, without external teachers or feedback from tools? We study this setting starting only from unlabeled seed questions with no ground-truth solutions, across three reasoning domains: math, science, and coding.
We propose Self-Verified Distillation, a simple post-training refinement algorithm in which the model generates candidate solutions to these seed questions, filters them using prompt-based self-verification, and trains on the resulting self-curated dataset.
Inspired by the UQ benchmark's use of multiple validators to screen candidate answers to hard unsolved questions, we adapt this validation-based filtering idea to self-training: the model filters its own generated solutions through a three-stage cascade of cycle-consistency, factuality, and correctness checks, accepting a solution only if it passes all stages with unanimous judge votes.
We find that sampling more candidate generations and using a larger verification budget during training data construction produces higher-quality self-curated data and, in turn, better reasoning models. We then train Qwen3 models at multiple scales with Self-Verified Distillation and obtain gains across all three domains. For Qwen3-4B, our method improves aggregate held-out pass@1 by +16. 7 points in math (AIME26 and HMMT), +11. 1 points in science ( and ), and +8.
3 points in coding (LCBv5 and LCBv6), with gains also extending to 0. 6B and 8B models. Compared to our test-time-only baseline (UQ-TTC), which improves performance by spending extra compute at inference time, Self-Verified Distillation achieves better performance in most settings while requiring only a single inference call at test time.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.