Interpreting Latent CoT Reasoning as Dynamical Systems
Quick Answer
This study interprets latent reasoning methods like CODI and COCONUT as dynamical systems, revealing structured dynamics with distinct stability classes.
Quick Take
This study interprets latent reasoning methods like CODI and COCONUT as dynamical systems, revealing structured dynamics with distinct stability classes. CODI acts as a stable attractor, while COCONUT is an unstable expanding system, with SIM-CoT supervision enhancing performance without altering dynamics.
Key Points
- Latent token sequences modeled as trajectories in representation space.
- Two stability classes identified: stable attractor (CODI) and unstable system (COCONUT).
- Quantitative measures include step-to-step change and Lyapunov sensitivity.
- SIM-CoT supervision enhances both models' performance without changing dynamics.
- Framework provides insights for improving latent reasoning performance.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Recent latent reasoning methods, such as CODI and COCONUT, face a fundamental interpretability problem: they maintain multiple superimposed candidate traces in the hidden space at each step, unlike explicit- CoT, which follows a single transparent reasoning trace. Existing mechanistic methods show compression, shortcuts, and superposition without explaining how reasoning evolves across latent steps. To address this gap, we model latent token sequences as trajectories in representation space and apply dynamical systems analysis to characterize the evolution of reasoning. Using quantitative measures, such as step-to-step change, direction consistency, and Lyapunov sensitivity, alongside qualitative projections, such as UMAP and DMD/PHATE, we show that latent CoT exhibits structured, non-random dynamics with two distinct stability classes. CODI behaves as a stable attractor, while COCONUT behaves as an unstable expanding system, and SIM-CoT supervision tightens both behaviors without changing the underlying dynamics. This framework advances the interpretability of latent CoT reasoning dynamics and provides actionable insights for improving latent reasoning performance. Code1 and Project page2 available online.
| Comments: | 15 pages, ICML 2026 FoGen Workshop |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2607.09698 [cs.AI] |
| (or arXiv:2607.09698v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2607.09698 arXiv-issued DOI via DataCite |
Submission history
From: Jerome Francis [view email]
[v1]
Sat, 20 Jun 2026 16:02:55 UTC (8,574 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →Adversarial Social Epistemology for Assemblies of Humans and Large Language Models
The paper introduces Adversarial Social Epistemology (ASE) to analyze how agents manipulate trust in public communications, highlighting mechanisms that undermine the reliability of testimony and inference. It critiques existing frameworks like epistemic bubbles and misinformation diffusion, proposing a new language for understanding trust breaches and auditing inferential chains in densely interactive environments involving humans and large language models.