Raon-Speech Technical Report
Quick Take
Raon-Speech is a 9B-parameter SpeechLM for English and Korean, achieving top performance on 42 benchmarks against models like Qwen2.5-Omni. Raon-SpeechChat extends this with full-duplex capabilities, trained on 119K hours of dialogue data, excelling in turn-taking and interruption-sensitive tasks.
Key Points
- Raon-Speech trained on 1.38M hours of curated English and Korean speech data.
- Achieved the strongest profile on speech-centric tasks among eight recent audio models.
- Raon-SpeechChat enables natural full-duplex conversations with advanced training techniques.
- Open-sourced all model checkpoints and an interactive demo for public use.
- Demonstrated competitive performance on full-duplex benchmarks, particularly in turn-taking.
Article Content
From source RSS / original summaryarXiv:2605. 23912v1 Announce Type: new Abstract: We present Raon-Speech, a top-performing 9B-parameter speech language model (SpeechLM) for English and Korean speech understanding, answering, and generation, and Raon-SpeechChat, a high-performing full-duplex extension for natural real-time conversation. Raon-Speech successfully transforms a pre-trained LLM into a SpeechLM that both understands and generates speech while preserving strong text capabilities. It trains on 1.
38M hours of highly curated English and Korean speech and text datasets with the following training stages: (1) speech modules alignment, (2) end-to-end SpeechLM pre-training with knowledge distillation, and (3) multi-task preference optimization-based post-training. Across 42 English and Korean speech and text benchmarks, Raon-Speech establishes the strongest overall profile on speech-centric tasks in our comparison against eight similarly sized recent audio foundation models, including Qwen2.
5-Omni and Fun-Audio-Chat, while preserving strong text question answering performance. Building upon it, Raon-SpeechChat enables natural full-duplex conversation by continual training on 119K hours of time-aligned real and synthetic dialogue data. It proceeds through three complementary training stages: (1) causal encoder adaptation, (2) full-duplex pre-training, (3) full-duplex fine-tuning for voice and role-control.
On multiple full-duplex benchmarks, Raon-SpeechChat shows its clearest strengths on the turn-taking and interruption-sensitive behaviors covered by FDB v1. 0, and remains competitive across the broader full-duplex evaluation suite. We open-source all model checkpoints, the training and inference pipeline, and an interactive demo.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?
The reliability of LLM judges for evaluating deep research agents is critically assessed using the REFLECT benchmark.