Polestar: Drift-Aware Cache Calibration and Token Commitment for Efficient Inference of Diffusion LLMs
Quick Answer
Polestar introduces a training-free inference framework that enhances diffusion large language models (dLLMs) by addressing token representation drift.
Quick Take
Polestar introduces a training-free inference framework that enhances diffusion large language models (dLLMs) by addressing token representation drift. It achieves up to 10.73% accuracy improvement and 3.7x higher throughput, setting new benchmarks in efficiency and decoding parallelism.
Key Points
- Polestar-Cache refreshes stale KV-cache positions for efficient reuse.
- Polestar-Commit detects drift events to identify commit-ready tokens.
- Achieves 3.67 tokens per forward pass, enhancing decoding parallelism.
- Sets new state-of-the-art on accuracy-throughput Pareto frontier.
- Demonstrates significant improvements across various dLLM families.
Paper Resources
📖 Reader Mode
~2 min readAbstract:The inference efficiency of diffusion large language models (dLLMs) is constrained by two challenges: bidirectional attention precludes efficient KV-cache reuse, while increasing decoding parallelism with static confidence thresholds can compromise generation quality. We observe that both challenges arise from a shared phenomenon: as tokens are decoded, their contextual integration through bidirectional attention causes token representations to drift (evolve) across decoding steps. This insight motivates Polestar, a training-free inference framework that uses token representation drift as a unified signal to jointly address both challenges. Polestar comprises two components: Polestar-Cache, which identifies stale KV-cache positions via drift and performs sparse KV-cache refreshes to enable efficient reuse, and Polestar-Commit, which detects sharp drift events to reliably identify commit-ready tokens. Across mathematics and coding benchmarks on several dLLM families, Polestar sets a new state of the art on the accuracy-throughput Pareto frontier, achieving up to 10.73% accuracy improvement, up to 3.7x higher throughput, and high decoding parallelism of 3.67 tokens per forward pass over existing baselines.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.14107 [cs.CL] |
| (or arXiv:2607.14107v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.14107 arXiv-issued DOI via DataCite |
Submission history
From: Akshat Ramachandran [view email]
[v1]
Thu, 7 May 2026 18:05:39 UTC (3,600 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Letting the Data Speak: Extracting Keywords from Crowdsourced Collections with AI
The study evaluates three NLP approaches—Named Entity Recognition, Keyword Extraction, and Topic Modelling—using the Their Finest Hour Online Archive to automate keyword extraction from crowdsourced WWII collections. Findings suggest that while NLP methods show promise, no single approach is sufficient, and ethical considerations in automated keyword extraction are crucial for responsible stewardship.