LaCache: Exact Caching and Precision-Adaptive Inference for Diffusion Large Language Models
Quick Answer
LaCache introduces a training-free acceleration framework for Diffusion Large Language Models (DLLMs), achieving a 1.3X speedup through lossless caching and mixed precision techniques.
Quick Take
By utilizing Lossless State Memoization (LSM) for intermediate results, it enables significant reductions in redundant computation, and when combined with existing methods, it can reach up to 40.2X speedup while maintaining task accuracy.
Key Points
- LaCache employs three types of caches: EmbedCache, RoPECache, and FACache.
- Achieves approximately 1.3X end-to-end speedup over vanilla DLLMs.
- Integrates FP8 quantization to alleviate memory-bandwidth bottlenecks.
- When combined with other methods, reaches up to 40.2X speedup.
- Maintains comparable task accuracy while improving performance.
DeepSignal Analysis
What happened
LaCache is a framework designed to enhance the efficiency of Diffusion Large Language Models (DLLMs) without requiring additional training. It achieves a 1.3X speedup through lossless caching and mixed precision techniques, and when combined with other methods, it can reach up to 40.2X speedup while maintaining task accuracy.
Key evidence
- LaCache employs Lossless State Memoization (LSM) to cache three types of intermediate results, which helps reduce redundant computations during the denoising steps of DLLMs.
- The framework integrates a per-group FP8 quantization strategy for feedforward layers, which is tailored to the activation distributions that vary across the diffusion process.
- Experiments indicate that LaCache alone provides approximately 1.3X end-to-end speedup compared to standard DLLMs, showcasing its potential for performance improvement.
Why it matters
The introduction of LaCache addresses a significant inefficiency in current DLLM implementations by minimizing operator-level redundancy. This can lead to faster processing times in applications that rely on large language models, making them more practical for real-time use cases. The ability to achieve substantial speedups while maintaining accuracy is crucial for developers and researchers looking to optimize model performance without compromising output quality.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Diffusion-based Large Language Models(DLLMs) enable parallel generation via Semi-Autoregressive (SAR) decoding in text generation. However, current methods suffer from severe operator-level redundancy: they recompute the entire sequence during denoising steps, ignoring that the prefix and masked suffix remain invariant within a block. We propose LaCache, a training-free acceleration framework that alleviates this redundancy through lossless caching and mixed precision. Specifically, LaCache employs Lossless State Memoization (LSM) by caching three types of intermediate results: (i) EmbedCache for embedding outputs, (ii) RoPECache for token-wise pre-attention states, and (iii) FACache for the online softmax statistics within FlashAttention. These caches allow the model to skip redundant computation on unchanged tokens without altering the output. To further alleviate memory-bandwidth bottlenecks, LaCache inegrates a per-group FP8 quantization strategy for FFN layers, tailored to step-dependent activation distributions across the diffusion process. Experiments demonstrate that LaCache alone achieves approximately 1.3X end-to-end speedup over vanilla DLLM. When combined with existing acceleration methods, LaCache reaches up to 40.2X end-to-end speedup while maintaining comparable task accuracy.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.16339 [cs.AI] |
| (or arXiv:2607.16339v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2607.16339 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xingru Chen [view email]
[v1]
Thu, 16 Jul 2026 10:49:15 UTC (3,782 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →HOBA: Hierarchical On-Policy Bidding Agents for Adaptive Online Advertising
HOBA (Hierarchical On-policy Bidding Agents) is a novel hierarchical reinforcement learning framework that enhances online advertising bidding systems by improving adaptability and reducing hyperparameter tuning costs. It utilizes a for hyperparameter inference, a SARSA agent for expert model selection, and a dynamic expert pool for bid execution, achieving a +3.6% increase in target cost during large-scale deployment and outperforming state-of-the-art baselines on AuctionNet.