EchoDistill:Alignment Noisy-to-Clean Self-Distillation for Robust Audio LLMs
Quick Answer
Echodistill introduces a self-distillation framework for Audio LLMs that enhances robustness against noise, achieving a 4.18% improvement in GSR under strong noise conditions.
Quick Take
By aligning noisy responses with a clean-audio teacher, it optimizes performance without additional inference costs, outperforming -only variants by up to 4.53% in GSR.
Key Points
- Echodistill leverages a frozen clean-audio teacher for semantic alignment.
- Achieves 4.18% average improvement in GSR under strong noise conditions.
- Outperforms GRPO-only variants by 3.02% in accuracy and 4.53% in GSR.
- No additional inference costs introduced in the self-distillation process.
- Extensive experiments validate the robustness of Audio against real-world noise.
Paper Resources
Article Content
From source RSS / original summaryarXiv:2605. 23954v1 Announce Type: new Abstract: Audio (ALLMs) are highly vulnerable to real-world noise, which often induces severe semantic drift and hallucinations. Existing robustness methods primarily rely on waveform-level acoustic enhancement, answer-level supervision, or the internal suppression of noise representations. To address these issues, we propose echodistill, an alignment-based noisy-to-clean self-distillation framework.
Echodistill leverages a frozen clean-audio teacher to provide semantic references for an inference-time noisy-audio student. Specifically, the student samples candidate responses under noisy conditions to expose its test-time behavior. These trajectories are then optimized via group-relative policy optimization (), where the token-level consistency with the teacher acts as a reward bonus.
By aligning the noisy student's candidate responses with clean semantic evidence, and applying audio-aware reward shaping, our method encourages reasoning trajectories that are both correct and genuinely acoustically grounded. Echodistill significantly improves the semantic reliability and task performance of Audio LLMs under complex noise, without introducing any additional inference costs. Extensive experiments show that: (I) Compared with the strongest baseline, echodistill achieves average improvements of 4.
18\%$\uparrow$ in GSR under strong noise. (II) Ablation results on Qwen-Omni further show that echodistill improves over the GRPO-only variant by 3. 02\%$\uparrow$ in Acc, 3. 89\%$\uparrow$ in Noisy, and 4. 53\%$\uparrow$ in GSR on average. Our codes are available at https://anonymous. 4open. science/r/echodistill-10DE.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.