Dynamic-dLLM: Dynamic Cache-Budget and Adaptive Parallel Decoding for Training-Free Acceleration of Diffusion LLM
Quick Answer
Dynamic-dLLM introduces a training-free framework that enhances inference efficiency of diffusion LLMs like LLaDA-8B-Instruct by over 3 times.
Quick Take
Dynamic-dLLM introduces a training-free framework that enhances inference efficiency of diffusion LLMs like LLaDA-8B-Instruct by over 3 times. It employs Dynamic Cache Updating and Adaptive Parallel Decoding to optimize performance on benchmarks such as and GSM8K, outperforming existing acceleration methods. This solution allows for efficient deployment without sacrificing model performance.
Key Points
- Dynamic Cache Updating allocates cache budgets based on token dynamics across layers.
- Adaptive Parallel Decoding calibrates thresholds for balancing quality and efficiency.
- Achieves over 3x speedup in inference while maintaining performance on various benchmarks.
- Outperforms state-of-the-art acceleration methods in efficiency and effectiveness.
- Code available for implementation at GitHub: https://github.com/TianyiWu233/DYNAMIC-DLLM.
Paper Resources
Article Content
From source RSS / original summaryarXiv:2606. 26120v1 Announce Type: new Abstract: Diffusion Large Language Models (dLLMs) offer a promising alternative to autoregressive models, excelling in text generation tasks due to their bidirectional attention mechanisms. However, their computational complexity scales on the order of L cubed with the sequence length L.
This poses significant challenges for long-sequence and real-time applications, primarily due to the lack of compatibility with key-value caching and the non-autoregressive nature of denoising steps. Existing acceleration methods rely on static caching or parallel decoding strategies, which fail to account for the dynamic behavior of token properties across layers and decoding steps.
We propose Dynamic-dLLM, a training-free framework that enhances dLLM inference efficiency through two components: Dynamic Cache Updating (DCU), which adaptively allocates cache-update budgets based on layer-wise token dynamics, and Adaptive Parallel Decoding (APD), which dynamically calibrates decoding thresholds to balance generation quality and efficiency. Extensive experiments on models like LLaDA-8B-Instruct, LLaDA-1.
5, and Dream-v0-7B-Instruct across benchmarks such as , GSM8K, and HumanEval demonstrate that Dynamic-dLLM significantly improves inference speed. It attains an average speedup exceeding 3 times while maintaining performance. Dynamic-dLLM outperforms state-of-the-art acceleration methods and provides a plug-and-play solution for efficient dLLM deployment without compromising performance. The code is available at https://github. com/TianyiWu233/DYNAMIC-DLLM.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Quantifying Prior Dominance in Systems
The study introduces the Normalized Context Utilization (NCU) metric to evaluate Retrieval-Augmented Generation (RAG) systems, revealing that Small Language Models (SLMs) outperform larger models in factual extraction. The findings indicate that traditional scaling laws yield diminishing returns, with a commercial API frequently failing against adversarial evidence due to systemic confidence collapse.