Fast-dLLM++: Fr\'{e}chet Profile Decoding for Faster Diffusion LLM Inference
Quick Answer
This paper shows that Fast-dLLM++ introduces Fréchet profile decoding for diffusion LLMs, enhancing throughput by up to 37% without altering existing models.
Quick Take
Fast-dLLM++ introduces Fréchet profile decoding for diffusion LLMs, enhancing throughput by up to 37% without altering existing models. This method leverages heterogeneous confidence profiles to improve parallel token generation, achieving better accuracy and efficiency on benchmarks like GSM8K and MATH.
Key Points
- Fast-dLLM++ enhances parallel token generation in diffusion LLMs.
- Achieves up to 37% higher throughput while maintaining accuracy.
- Utilizes heterogeneous confidence profiles for improved decoding.
- Compatible with existing Fast-dLLM models and processes.
- Empirical gains demonstrated on benchmarks like GSM8K and MATH.
Paper Resources
Article Content
From source RSS / original summaryarXiv:2606. 02955v1 Announce Type: new Abstract: Diffusion large language models promise parallel token generation, yet inference remains bottlenecked by deciding which masked tokens can be safely committed together. Fast-dLLM addressed this with KV caching and confidence-guided parallel decoding, but its decoding theory uses a homogeneous high-confidence assumption that effectively reduces each candidate set to its weakest selected token.
We argue that this leaves speed on the table because real decoding steps exhibit heterogeneous confidence profiles. We propose \textbf{Fast-dLLM++}, a training-free extension that introduces \emph{Fr\'{e}chet profile decoding}: selecting parallel commit sets from the full sorted confidence profile rather than a single worst-case confidence.
The resulting rule is a heterogeneous-confidence generalization of Fast-dLLM's factor selector and it recovers the previous rule exactly in the equal-confidence case and adds a provable \emph{heterogeneity bonus} when the selected tokens have uneven confidences. Fast-dLLM++ leaves the model, diffusion process, and cache implementation entirely unchanged, making it a drop-in replacement for existing Fast-dLLM decoding.
Experiments on GSM8K, MATH, HumanEval, and MBPP with the LLaDA-8B model show that the theoretical improvement translates directly into empirical gains: profile-aware selection improves the accuracy--throughput frontier by exploiting safe parallelism that weakest-token rules miss, achieving up to 37\% higher throughput at comparable accuracy. Our anonymous code release is at https://github. com/Ringo-Star/FastdLLM_plusplus.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Letting the Data Speak: Extracting Keywords from Crowdsourced Collections with AI
The study evaluates three NLP approaches—Named Entity Recognition, Keyword Extraction, and Topic Modelling—using the Their Finest Hour Online Archive to automate keyword extraction from crowdsourced WWII collections. Findings suggest that while NLP methods show promise, no single approach is sufficient, and ethical considerations in automated keyword extraction are crucial for responsible stewardship.