DecodeShare: Tracing the Shared Subspace of LLM Decode-Time Decisions
Quick Answer
This paper shows that The DecodeShare protocol identifies a low-dimensional shared subspace in LLMs' decode-time hidden states, significantly impacting decision performance.
Quick Take
Disturbing this subspace degrades performance more than altering prefill-derived or random subspaces. This discovery enhances activation steering reliability for downstream applications.
Key Points
- DecodeShare identifies a shared subspace in during decode time.
- Disturbing this subspace degrades performance significantly more than other subspaces.
- The shared subspace aids in reliable activation steering for LLMs.
- Evaluating steering vectors at decode time yields better results than prefill proxies.
- Code for DecodeShare is publicly available for further research.
DeepSignal Analysis
What happened
The DecodeShare protocol identifies a low-dimensional shared subspace in the hidden states of large language models (LLMs) during decoding. Disturbing this subspace significantly reduces decision performance compared to altering prefill-derived or random subspaces. This finding suggests that the shared subspace plays a crucial role in the reliability of activation steering for various applications.
Key evidence
- The DecodeShare protocol identifies a low-dimensional shared subspace in LLMs' decode-time hidden states.
- Experiments show that disturbing the shared subspace degrades decision performance more than altering prefill-derived or random subspaces.
- The shared subspace can serve as a high-leverage causal channel at decode time, enhancing activation steering reliability.
Why it matters
Understanding the shared subspace in LLMs can improve the performance of various AI applications by providing insights into how these models make decisions during decoding. This knowledge can lead to better activation steering methods, which are essential for optimizing model outputs in real-world tasks. The implications of this research could influence the design of future LLMs and their applications.
Paper Resources
📖 Reader Mode
~2 min readAuthors:Zishan Shao, Lixun Zhang, Kangning Cui, Yixiao Wang, Ting Jiang, Hancheng Ye, Qinsi Wang, Zhixu Du, Yuzhe Fu, Fan Yang, Danyang Zhuo, Yiran Chen, Hai Helen Li
Abstract:Large language models (LLMs) handle many tasks with one set of parameters, but under KV-cached inference it is unclear what task-general structure, if any, is used at decode time rather than during prefill. We propose DecodeShare, a protocol that identifies a low-dimensional subspace consistently shared across tasks in decode-time hidden states, and then tests its causal role by removing that subspace only during decoding. In our experiments, disturbing the discovered shared subspace degrades decision performance far more than disturbing either a prefill-derived or random subspace under the same intervention budget. We further show this decode-shared subspace has practical consequences for activation steering: common steering directions can overlap the task-general decode channel. Projecting out this shared subspace directly separates the functional roles of the two components, while evaluating steering vectors at decode-time yields more reliable signal for downstream deployment than prefill-based proxies. Despite its compactness, the shared subspace can serve as a high-leverage causal channel at decode time. Code is available at: this https URL.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.20469 [cs.AI] |
| (or arXiv:2607.20469v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2607.20469 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zishan Shao [view email]
[v1]
Thu, 21 May 2026 18:59:00 UTC (1,321 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →HOBA: Hierarchical On-Policy Bidding Agents for Adaptive Online Advertising
HOBA (Hierarchical On-policy Bidding Agents) is a novel hierarchical reinforcement learning framework that enhances online advertising bidding systems by improving adaptability and reducing hyperparameter tuning costs. It utilizes a for hyperparameter inference, a SARSA agent for expert model selection, and a dynamic expert pool for bid execution, achieving a +3.6% increase in target cost during large-scale deployment and outperforming state-of-the-art baselines on AuctionNet.