Value-Informed Schedule Trimming for Accelerated Sequential Monte Carlo Guidance in Discrete Diffusion
Quick Answer
VISTA is a training-free method for optimizing sparse guidance in Sequential Monte Carlo (SMC) for discrete diffusion tasks, achieving significant speedups.
Quick Take
It reduces post-selection latency from 204.23s to 38.16s while maintaining high performance, with a 0.99 unique-toxic ratio in toxic-text generation, closely matching full guidance results.
Key Points
- VISTA selects guided timesteps under a fixed budget without training.
- Achieves 5.35x speedup in post-selection latency for toxic-text generation.
- Optimizes empirical surrogate using value statistics from warmup generations.
- Outperforms hand-designed sparse schedules across various tasks.
- Maintains similar guidance complexity as other sparse schedules.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Reward-guided discrete diffusion enables test-time alignment with downstream objectives, but applying Sequential Monte Carlo (SMC) guidance at every denoising step is computationally expensive. Active guidance steps require additional reward evaluations and, for gradient-based proposals, value-gradient computations. Sparse guidance reduces this cost by guiding only a subset of timesteps, but commonly used schedules are fixed heuristics whose effectiveness can vary substantially across tasks. We propose VISTA, a training-free method that automatically selects guided timesteps under a fixed budget. VISTA formulates sparse guidance as approximation of the ideal full-guidance probability path and derives an estimable upper bound on \(\ell_1\) path deviation. Using value statistics collected from a small number of full-guidance warmup generations, VISTA optimizes the resulting empirical surrogate exactly via dynamic programming. The warmup generations can be retained as model outputs, and after schedule selection VISTA has the same per-generation guidance complexity as other sparse schedules with the same budget. Across text, biological sequence, and image generation tasks, VISTA discovers task-dependent schedules and outperforms hand-designed sparse schedules in most tested settings, while approaching or exceeding full-guidance performance on several task metrics. In toxic-text generation, for example, VISTA attains a 0.99 unique-toxic ratio compared with 1.00 for full guidance while reducing post-selection per-generation latency from 204.23 s to 38.16 s, a 5.35$\times$ speedup.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.20669 [cs.AI] |
| (or arXiv:2610.20669v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20669 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jaehyeon Kim [view email]
[v1]
Wed, 14 Oct 2026 10:49:59 UTC (1,419 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →VEPA: Verified Evolving Formal Policy for Lifelong Agent Guardrails
VEPA introduces a framework for verified evolving formal policies for LLM agents, enhancing safety by adapting guardrails based on deployment feedback. It outperforms existing methods across benchmarks like ASSEBench-Safety and ASSEBench-Security, ensuring deterministic policy enforcement even under challenging conditions.