SLPO: Scaling Latent Reasoning via a Surrogate Policy
Quick Answer
SLPO introduces Surrogate Latent Policy Optimization to enhance outcome-reward reinforcement learning for autoregressive latent reasoners, improving Pass@$k$ performance and enabling longer computations for complex tasks.
Quick Take
This method addresses the limitations of latent reasoning by providing a trajectory-level credit assignment and a correctness-supervised stopping mechanism, leading to better efficiency in reasoning tasks.
Key Points
- SLPO improves Pass@$k$ performance in autoregressive latent reasoning tasks.
- Introduces trajectory-level credit assignment for better outcome-reward optimization.
- Enables longer latent computations for harder instances with higher accuracy.
- Addresses limitations of latent reasoning by incorporating a stopping mechanism.
- Demonstrates effectiveness in both continuous and soft thinking settings.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Reinforcement learning with verifiable rewards has become the predominant recipe for eliciting test-time scaling in explicit Chain-of-Thought reasoners. Yet this scaling path remains computationally costly, since every intermediate step must be decoded as a language token. Latent reasoning instead carries intermediate computation as continuous vectors and already matches or surpasses explicit CoT at far shorter horizons. Despite this promise, latent reasoners remain largely imitation-bound, while explicit CoT has already moved past imitation via outcome-reward RL. Latent trajectories lack a tractable per-step likelihood and an adaptive stopping interface under fixed thinking budgets, so outcome rewards cannot elicit latent test-time scaling. We introduce Surrogate Latent Policy Optimization (SLPO) to bring outcome-reward RL to autoregressive latent reasoners: an empirical surrogate policy density over latent transitions for trajectory-level credit assignment, and a correctness-supervised stopping head that outcome-reward optimization refines into a variable-horizon policy. Across continuous and soft thinking settings, SLPO improves Pass@$k$ under parallel sampling and allocates longer latent computation to harder instances with higher deterministic accuracy.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2607.19691 [cs.CL] |
| (or arXiv:2607.19691v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.19691 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Runyang You [view email]
[v1]
Wed, 22 Jul 2026 02:45:08 UTC (2,983 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.