Reinforcement Learning for Data-Efficient Code-Switched ASR
Quick Answer
The study introduces a reinforcement learning approach for adapting audio-language models to code-switched ASR, achieving performance comparable to full dataset fine-tuning with only 10% of the data.
Quick Take
The study introduces a reinforcement learning approach for adapting audio-language models to code-switched ASR, achieving performance comparable to full dataset fine-tuning with only 10% of the data. Using Qwen2-Audio, the method effectively reduces translation errors and script contamination, especially for typologically distant language pairs, and demonstrates zero-shot transfer to human-recorded corpora.
Key Points
- Introduces reinforcement learning with verifiable rewards for code-switched ASR adaptation.
- Achieves performance matching full dataset fine-tuning using only 10% of the data.
- Error rate and script fidelity rewards significantly reduce translation errors and contamination.
- Demonstrates effectiveness across 10 language pairs, particularly distant typological pairs.
- Zero-shot transfer to human-recorded code-switching corpus validated.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Audio-language models can be prompted for code-switched speech, but their decoding is not optimized for code-switching and often fails at language boundaries. We propose a practical reinforcement learning with verifiable rewards recipe for data-efficient adaptation of audio-language models to code-switched ASR using group relative policy optimization, combining an error rate reward with a script fidelity reward that penalizes wrong writing systems and a two-pass draft-and-refinement procedure. Using Qwen2-Audio as a reproducible testbed across 10 language pairs, training on only TTS code-switched speech, we show that RLVR with 10% of the data matches LoRA supervised fine-tuning trained on the full dataset, with the largest gains on typologically distant pairs. The error rate reward eliminates translation errors while the script fidelity reward separately reduces script contamination without degradation. These gains transfer zero-shot to a human-recorded code-switching corpus.
| Comments: | Accepted at Interspeech 2026 |
| Subjects: | Computation and Language (cs.CL); Sound (cs.SD) |
| Cite as: | arXiv:2607.02757 [cs.CL] |
| (or arXiv:2607.02757v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.02757 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Peter Vickers [view email]
[v1]
Thu, 2 Jul 2026 20:44:53 UTC (5,114 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Quantifying Prior Dominance in Systems
The study introduces the Normalized Context Utilization (NCU) metric to evaluate Retrieval-Augmented Generation (RAG) systems, revealing that Small Language Models (SLMs) outperform larger models in factual extraction. The findings indicate that traditional scaling laws yield diminishing returns, with a commercial API frequently failing against adversarial evidence due to systemic confidence collapse.