Revisiting the Relation Between Language Model Perplexity and ASR Word Error Rate for Modern End-to-End Speech Recognition
Quick Answer
This paper reevaluates the link between language model perplexity (PPL) and automatic speech recognition (ASR) word error rate (WER) in modern end-to-end systems, revealing that internal language modeling capacity alters the previously linear relationship.
Quick Take
It also examines the impact of external language models and encoder context length on this relationship, suggesting that internal language modeling must be factored into performance assessments.
Key Points
- Internal language modeling in ASR systems affects the PPL-WER relationship significantly.
- The study questions the linearity of the PPL-WER relationship in modern ASR.
- External language models may still enhance performance in end-to-end ASR systems.
- Encoder context length plays a crucial role in the PPL-WER dynamics.
- The paper highlights the importance of internal language modeling in performance evaluations.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Language model (LM) perplexity (PPL) has historically been used as a proxy for automatic speech recognition (ASR) word error rate (WER), with prior work reporting an approximately linear relation in log-log space. Modern end-to-end ASR systems challenge this assumption because they already contain internal language modeling capacity, are often evaluated without external language models, and can now be combined with neural LMs and large language models (LLMs) through different recognition strategies. This paper revisits the relation between PPL and WER for modern ASR systems. We study whether external LMs still improve current end-to-end ASR systems, whether the PPL-WER relation remains linear in log-log space, how encoder context length affects this relation, and how LLM perplexities fit into the trend observed for standard neural LMs. We further investigate internal language modeling (ILM) in attention-based encoder-decoder systems and show that ILM subtraction changes the observed PPL-WER relation, indicating that the decoder's internal LM must be considered when interpreting the effect of external LM quality.
| Comments: | Submitted to SLT 2026 |
| Subjects: | Computation and Language (cs.CL); Audio and Speech Processing (eess.AS) |
| Cite as: | arXiv:2607.05612 [cs.CL] |
| (or arXiv:2607.05612v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.05612 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Mohammad Zeineldeen [view email]
[v1]
Mon, 6 Jul 2026 20:15:49 UTC (197 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.