Steering Follows Geometry, Not Labels: Emotion Directions in a Full-Duplex Speech Model
Quick Answer
The study explores emotion steering in the open-source full-duplex speech model Moshi, demonstrating that emotions can be linearly decoded but activation steering varies by emotion type.
Quick Take
Happy, angry, and surprise emotions share a common direction, while sadness is distinctly steerable. This method requires minimal computational cost, involving only a few vector additions per frame without retraining.
Key Points
- Moshi is an open-source full-duplex speech model focused on emotion modulation.
- Emotion steering achieved with mean-difference activation steering, costing minimal computational resources.
- Happy, angry, and surprise emotions steer towards a shared direction, unlike sadness.
- Emotion control in full-duplex models is less explored compared to TTS and turn-based systems.
- Study accepted at NeurIPS 2026 Workshop on Real-Time Conversational Agents.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Full-duplex voice agents need to modulate emotion and delivery during real-time conversations, when de-escalating a complaint, carrying urgency in dispatch, softening a clinical result. Emotion and delivery control is well studied for TTS and turn based models through prompt-conditioned synthesis, reference-conditioned synthesis and activation steering; PersonaPlex controls identity in a duplex model but not affect.
We study emotion steering in Moshi, a fully open sourced full-duplex speech language model, across four emotions, using mean-difference activation steering, which costs only a few vector additions per frame and no retraining.
We show that emotion is linearly decodable from Moshi's residual stream, but activation steering is only partially achievable, and unevenly so; as happy, angry and surprise steer towards a shared direction while sad is distinctly steerable. We also show that the shared component across the three emotions cannot simply be projected away from all the emotions equally.
| Comments: | Accepted at the NeurIPS 2026 Workshop on Real-Time Conversational Agents (RTCA), Sydney. OpenReview: this https URL |
| Subjects: | Computation and Language (cs.CL); Sound (cs.SD) |
| ACM classes: | I.2.7; I.2.6 |
| Cite as: | arXiv:2610.08887 [cs.CL] |
| (or arXiv:2610.08887v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.08887 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Pulak Kuli [view email]
[v1]
Tue, 6 Oct 2026 15:29:37 UTC (53 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.