Revisiting the Relation Between Language Model Perplexity and ASR Word Error Rate for Modern End-to-End Speech Recognition
Quick Answer
This paper reevaluates the link between language model perplexity (PPL) and automatic speech recognition (ASR) word error rate (WER) in modern end-to-end systems, revealing that internal language modeling capacity alters the previously linear relationship.
Quick Take
This paper reevaluates the link between language model perplexity (PPL) and automatic speech recognition (ASR) word error rate (WER) in modern end-to-end systems, revealing that internal language modeling capacity alters the previously linear relationship. It also examines the impact of external language models and encoder context length on this relationship, suggesting that internal language modeling must be factored into performance assessments.
Key Points
- Internal language modeling in ASR systems affects the PPL-WER relationship significantly.
- The study questions the linearity of the PPL-WER relationship in modern ASR.
- External language models may still enhance performance in end-to-end ASR systems.
- Encoder context length plays a crucial role in the PPL-WER dynamics.
- The paper highlights the importance of internal language modeling in performance evaluations.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Language model (LM) perplexity (PPL) has historically been used as a proxy for automatic speech recognition (ASR) word error rate (WER), with prior work reporting an approximately linear relation in log-log space. Modern end-to-end ASR systems challenge this assumption because they already contain internal language modeling capacity, are often evaluated without external language models, and can now be combined with neural LMs and large language models (LLMs) through different recognition strategies. This paper revisits the relation between PPL and WER for modern ASR systems. We study whether external LMs still improve current end-to-end ASR systems, whether the PPL-WER relation remains linear in log-log space, how encoder context length affects this relation, and how LLM perplexities fit into the trend observed for standard neural LMs. We further investigate internal language modeling (ILM) in attention-based encoder-decoder systems and show that ILM subtraction changes the observed PPL-WER relation, indicating that the decoder's internal LM must be considered when interpreting the effect of external LM quality.
| Comments: | Submitted to SLT 2026 |
| Subjects: | Computation and Language (cs.CL); Audio and Speech Processing (eess.AS) |
| Cite as: | arXiv:2607.05612 [cs.CL] |
| (or arXiv:2607.05612v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.05612 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Mohammad Zeineldeen [view email]
[v1]
Mon, 6 Jul 2026 20:15:49 UTC (197 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Quantifying Prior Dominance in Systems
The study introduces the Normalized Context Utilization (NCU) metric to evaluate Retrieval-Augmented Generation (RAG) systems, revealing that Small Language Models (SLMs) outperform larger models in factual extraction. The findings indicate that traditional scaling laws yield diminishing returns, with a commercial API frequently failing against adversarial evidence due to systemic confidence collapse.