PiDA: Phonetically-Informed Data Augmentation for Robust Vietnamese Speech Translation
Quick Answer
The study introduces Phonetically-Informed Data Augmentation (PiDA) to enhance Vietnamese speech translation by addressing ASR substitution errors, achieving up to +2.04 BLEU improvement on erroneous outputs.
Quick Take
This method leverages phonetic embeddings to generate realistic corruptions, significantly boosting Neural Machine Translation performance.
Key Points
- First systematic categorization of ASR errors in Vietnamese speech translation.
- Phonetic confusions, not random noise, primarily cause ASR substitution errors.
- PiDA improves translation quality on erroneous ASR outputs by up to +2.04 BLEU.
- Fine-tuning on PiDA-augmented data also enhances clean-text performance.
- Utilizes phonetic word embeddings for generating ASR-like corruptions.
Paper Resources
Source Excerpt
arXiv:2606. 12911v1 Announce Type: new Abstract: Cascaded speech translation (ST) systems suffer from error propagation when Automatic Speech Recognition (ASR) outputs incorrect transcripts. We present the first systematic categorization of ASR errors for Vietnamese ST, classifying substitution errors by phonetic cause and quantifying their impact on downstream Neural Machine Translation (NMT) performance using Linear Mixed-Effects Modelling.
We confirm that most ASR substitution errors arise from phonetic confusions rather than random noise, and that these phonetic errors significantly degrade ST quality. …
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.