FlowLM: Few-Step Language Modeling via Diffusion-to-Flow Adaptation
8h ago
Quick Take
FlowLM transforms diffusion models for efficient few-step language generation with superior quality.
Key Points
- FlowLM achieves high-quality generation with few training epochs.
- Outperforms 2,000-step diffusion sampling in quality.
- Introduces a new objective for effective flow matching.
📖 Reader Mode
~2 min readAbstract:We present FlowLM, a flow matching language model transformed from pre-trained diffusion language models via efficient fine-tuning. By re-aligning the curved sampling trajectories of diffusion models into straight-line flows, FlowLM enables high quality few-step generation that rivals or even outperforms the quality of 2,000-step diffusion sampling with very few training epochs. Remarkably, finetuned FlowLM reaches performance saturation with only half as many training epochs as training from scratch, both approaches greatly outperforming the original diffusion model, thereby validating our method. Furthermore, we validate a more effective training objective for flow matching: predicting clean data to consistently guide the sampling process towards the true data distribution. Empirical results demonstrate that our approach is highly effective for high-quality, few-step text generation.
| Comments: | 26 pages, 11 figures |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.20199 [cs.CL] |
| (or arXiv:2605.20199v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2605.20199 arXiv-issued DOI via DataCite |
Submission history
From: Runzhe Zhang [view email]
[v1]
Mon, 6 Apr 2026 10:36:22 UTC (3,537 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →arXiv cs.CL·Leyao Wang, Yanan He, Peng Chen, Asaf Yehudai, Yixin Liu, Rex Ying, Michal Shmueli-Scheuer, Arman Cohan
1d ago FeaturedOriginal
Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?
AI Summary
The reliability of LLM judges for evaluating deep research agents is critically assessed using the REFLECT benchmark.