Residualized Temporal Sparse Autoencoders for Interpreting Diffusion Models
Quick Take
The paper introduces residualized temporal sparse autoencoders (SAEs) for interpreting diffusion model activations, enhancing understanding of feature trajectories over time. By analyzing full activation trajectories instead of individual timesteps, the approach captures complex structures beyond linear predictability, demonstrating its effectiveness on Stable Diffusion 1.5 through various studies.
Key Points
- Residualized temporal SAEs analyze full activation trajectories from diffusion models.
- The method captures complex structures beyond linear predictability.
- Demonstrated effectiveness through reconstruction and ablation studies.
- Qualitative steering experiments conducted on Stable Diffusion 1.5.
- Provides a framework for studying temporally structured diffusion activations.
Article Content
From source RSS / original summaryarXiv:2605. 27813v1 Announce Type: new Abstract: Text-to-image diffusion models generate images through an iterative denoising process, so internal neural layers produce trajectories of activations rather than single static representations. Sparse autoencoders (SAEs) have recently been used to decompose diffusion activations into interpretable feature directions, but most approaches analyze activations at individual timesteps or condition on time rather than learning directly from full activation trajectories.
In this work, we introduce residualized temporal SAEs for diffusion activation trajectories. We collect activations across denoising time, fit linear predictors between neighboring timesteps, and represent each trajectory using an initial activation together with residual components not explained by these linear dynamics. Training an SAE on this residualized representation encourages sparse latents to capture structure beyond what is linearly predictable.
The residualized decoder directions can be mapped back into activation space, allowing each latent to be analyzed as a feature trajectory over denoising time. Through reconstruction and ablation studies, spatiotemporal feature analysis, and qualitative steering experiments on Stable Diffusion~1. 5, we show that residualized temporal SAEs provide a useful framework for studying temporally structured diffusion activations.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →Evi-Steer: Learning to Steer Biomedical Vision-Language Models through Efficient and Generalizable Evidential Tuning
Evi-Steer introduces a novel evidential tuning framework for BiomedCLIP, achieving 0.11% parameter updates while enhancing uncertainty-aware fine-tuning. It outperforms state-of-the-art methods across 15 biomedical imaging datasets, proving effective in few-shot learning and domain shifts for clinical applications.