Dynamic-in-Few-Step: Unifying Dynamic Computation and Few-Step Distillation for Efficient Video Generation
Quick Answer
The proposed Dynamic-in-Few-Step framework integrates dynamic structural sparsification into video diffusion model distillation, achieving a 30x speedup while maintaining competitive quality.
Quick Take
The proposed Dynamic-in-Few-Step framework integrates dynamic structural sparsification into video diffusion model distillation, achieving a 30x speedup while maintaining competitive quality. This method reduces per-step FLOPs by 24% on Wan-14B, enhancing inference efficiency with a 1.2x wall-clock gain over traditional 4-step distillation.
Key Points
- Dynamic-in-Few-Step optimizes denoising steps and model sparsity jointly.
- The framework transforms pre-trained VDMs into compact, step-specific Mixture-of-Models.
- Progressive Training Strategy ensures coherent learning across timesteps.
- Achieves a 30x speedup over a 50-step teacher model.
- Specialized inference engine enhances deployment efficiency.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Video Diffusion Models (VDMs) have demonstrated superior generation quality but suffer from prohibitive computational costs. While recent few-step distillation techniques significantly accelerate inference, they typically enforce a static model architecture across all denoising stages, ignoring the varying computational demands inherent to different noise levels. In this work, we propose a novel post-training acceleration framework that exploits this redundancy by integrating dynamic structural sparsification directly into the distillation process. Unlike conventional post-hoc compression applied to a fixed diffusion pipeline, our approach jointly optimizes the denoising steps and structured model sparsity, transforming a pre-trained VDM into a compact, step-specific Mixture-of-Models (MoM). To address the training instability arising from this joint optimization, we introduce a Progressive Training Strategy coupled with an Output Rollout Mechanism, which ensures the coherent learning of structural decisions across timesteps. Furthermore, we develop a specialized inference engine to deploy the resulting MoM efficiently. Our method is orthogonal to existing acceleration techniques and highly effective: On Wan-14B, it removes 24% of the per-step FLOPs on top of 4-step distillation, adding a 1.2x wall-clock gain and reaching a 30x speedup over the 50-step teacher while preserving competitive generation quality.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2607.06631 [cs.CV] |
| (or arXiv:2607.06631v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2607.06631 arXiv-issued DOI via DataCite |
Submission history
From: Yu Cheng [view email]
[v1]
Tue, 7 Jul 2026 13:14:33 UTC (5,732 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
ProMoE-FL introduces a Prototype-conditioned Mixture-of-Experts framework for multimodal federated learning, effectively addressing missing modalities. It outperforms existing methods on four chest X-ray datasets, demonstrating superior feature synthesis capabilities in both homogeneous and heterogeneous settings.