Unified Backbone Refinement for Diffusion Models via Internal-Latent Analysis
Quick Answer
The paper introduces DUNE, a training-free refinement framework for diffusion models that enhances fidelity and reduces hallucinations by detecting and suppressing abrupt fluctuations in deep latents.
Quick Take
The paper introduces DUNE, a training-free refinement framework for diffusion models that enhances fidelity and reduces hallucinations by detecting and suppressing abrupt fluctuations in deep latents. This method, applicable to both U-Net and Transformer architectures, offers insights into controlling diffusion backbones effectively.
Key Points
- DUNE improves fidelity while reducing hallucinations in diffusion models.
- The framework is training-free and uses an EMA-based criterion for detection.
- Applicable to both U-Net and Transformer-based diffusion architectures.
- Findings reveal that early-stage fluctuations in latents cause artifacts.
- Extensive experiments validate DUNE's effectiveness across multiple backbones.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Diffusion models have achieved remarkable success across diverse domains, with performance closely related to the denoising backbones that parameterize the score function. In this paper, we present a systematic, phase-aware analysis of diffusion components and show that abrupt, early-stage fluctuations in deep latents are strongly associated with artifacts. Guided by these findings, we introduce DUNE (Diffusion Unified Network refiNEr), a training-free refinement framework that detects abrupt deviations in deep low-noise internal latents using a shared EMA-based criterion, and applies backbone-specific suppression to the detector-selected entries. Although derived from U-Net, the same detect-suppress principle extends naturally to Transformer-based diffusion models by acting on the latents of deep self-attention blocks. Extensive experiments across multiple backbones indicate that DUNE improves fidelity while reducing hallucinations, offering new insight into where and when diffusion backbones should be controlled.
| Comments: | 45 pages, 23 figures. Accepted at the European Conference on Computer Vision (ECCV) 2026 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2607.09753 [cs.CV] |
| (or arXiv:2607.09753v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2607.09753 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Haksoo Lim [view email]
[v1]
Sat, 4 Jul 2026 13:24:14 UTC (47,953 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
ProMoE-FL introduces a Prototype-conditioned Mixture-of-Experts framework for multimodal federated learning, effectively addressing missing modalities. It outperforms existing methods on four chest X-ray datasets, demonstrating superior feature synthesis capabilities in both homogeneous and heterogeneous settings.