SLPO: Scaling Latent Reasoning via a Surrogate Policy
Quick Answer
SLPO introduces Surrogate Latent Policy Optimization to enhance outcome-reward reinforcement learning for autoregressive latent reasoners, improving Pass@$k$ performance and enabling longer computations for complex tasks.
Quick Take
This method addresses the limitations of latent reasoning by providing a trajectory-level credit assignment and a correctness-supervised stopping mechanism, leading to better efficiency in reasoning tasks.
Key Points
- SLPO improves Pass@$k$ performance in autoregressive latent reasoning tasks.
- Introduces trajectory-level credit assignment for better outcome-reward optimization.
- Enables longer latent computations for harder instances with higher accuracy.
- Addresses limitations of latent reasoning by incorporating a stopping mechanism.
- Demonstrates effectiveness in both continuous and soft thinking settings.
Paper Resources
Source Excerpt
Reinforcement learning with verifiable rewards has become the predominant recipe for eliciting in explicit Chain-of-Thought reasoners. Yet this scaling path remains computationally costly, since every intermediate step must be decoded as a language token. Latent reasoning instead carries intermediate computation as continuous vectors and already matches or surpasses explicit CoT at far shorter horizons. Despite this promise, latent reasoners remain largely imitation-bound, whil
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.