Why Limit the Residual Stream to Layers and Not Tokens? Persistent Memory for Continuous Latent Reasoning
Quick Answer
The AGCLR model enhances the CoCoNuT paradigm by introducing a Gated Concept Stream, addressing the concept bottleneck in LLMs.
Quick Take
This innovation allows for persistent memory across reasoning passes, leading to improved performance on benchmarks like GSM8K and HotpotQA, with AGCLR outperforming vanilla CoCoNuT by resolving critical fact loss during reasoning. Code is available for further exploration.
Key Points
- AGCLR introduces a Gated Concept Stream to retain critical facts during reasoning.
- Performance improved on GSM8K and HotpotQA benchmarks compared to vanilla CoCoNuT.
- Concept bottleneck identified as a limitation in existing reasoning methods.
- Three learned gates (write, read, forget) control memory management in AGCLR.
- Code for AGCLR is publicly available for further research and application.
Paper Resources
Source Excerpt
arXiv:2606. 07720v1 Announce Type: new Abstract: (LLMs) have demonstrated remarkable reasoning abilities on mathematical and multi-hop planning tasks. The CoCoNuT (Chain of Continuous Thought) paradigm~\cite{hao2024coconut} extends this by enabling models to reason in latent space, exploring multiple reasoning paths simultaneously rather than committing to a single chain early on. However, we identify a limitation we term the \textbf{concept bottleneck}.
At each reasoning pass, intermediate hidden states are overwritten, causing the model to lose critical facts computed in earlier steps as reasoning depth increases. We observe this empirically. …
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →AINTMA: Agentic AI Architecture for Autonomous Test Management with Generative Intelligence, Secure Cloud Communication and Adaptive Quality Analytics
AINTMA, an autonomous test management architecture utilizing six specialized AI agents, achieves 88.4% test prioritization accuracy and reduces defect escape rates from 8.3% to 2.1%. The system demonstrates a 340% ROI within nine months, showcasing the potential of agentic AI in enhancing software quality management in cloud environments.