EchoDistill:Alignment Noisy-to-Clean Self-Distillation for Robust Audio LLMs
Quick Answer
Echodistill introduces a self-distillation framework for Audio LLMs that enhances robustness against noise, achieving a 4.18% improvement in GSR under strong noise conditions.
Quick Take
Echodistill introduces a self-distillation framework for Audio LLMs that enhances robustness against noise, achieving a 4.18% improvement in GSR under strong noise conditions. By aligning noisy responses with a clean-audio teacher, it optimizes performance without additional inference costs, outperforming GRPO-only variants by up to 4.53% in GSR.
Key Points
- Echodistill leverages a frozen clean-audio teacher for semantic alignment.
- Achieves 4.18% average improvement in GSR under strong noise conditions.
- Outperforms GRPO-only variants by 3.02% in accuracy and 4.53% in GSR.
- No additional inference costs introduced in the self-distillation process.
- Extensive experiments validate the robustness of Audio LLMs against real-world noise.
Paper Resources
Article Content
From source RSS / original summaryarXiv:2605. 23954v1 Announce Type: new Abstract: Audio Large Language Models (ALLMs) are highly vulnerable to real-world noise, which often induces severe semantic drift and hallucinations. Existing robustness methods primarily rely on waveform-level acoustic enhancement, answer-level supervision, or the internal suppression of noise representations. To address these issues, we propose echodistill, an alignment-based noisy-to-clean self-distillation framework.
Echodistill leverages a frozen clean-audio teacher to provide semantic references for an inference-time noisy-audio student. Specifically, the student samples candidate responses under noisy conditions to expose its test-time behavior. These trajectories are then optimized via group-relative policy optimization (GRPO), where the token-level consistency with the teacher acts as a reward bonus.
By aligning the noisy student's candidate responses with clean semantic evidence, and applying audio-aware reward shaping, our method encourages reasoning trajectories that are both correct and genuinely acoustically grounded. Echodistill significantly improves the semantic reliability and task performance of Audio LLMs under complex noise, without introducing any additional inference costs. Extensive experiments show that: (I) Compared with the strongest baseline, echodistill achieves average improvements of 4.
18\%$\uparrow$ in GSR under strong noise. (II) Ablation results on Qwen-Omni further show that echodistill improves over the GRPO-only variant by 3. 02\%$\uparrow$ in Acc, 3. 89\%$\uparrow$ in Noisy, and 4. 53\%$\uparrow$ in GSR on average. Our codes are available at https://anonymous. 4open. science/r/echodistill-10DE.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Letting the Data Speak: Extracting Keywords from Crowdsourced Collections with AI
The study evaluates three NLP approaches—Named Entity Recognition, Keyword Extraction, and Topic Modelling—using the Their Finest Hour Online Archive to automate keyword extraction from crowdsourced WWII collections. Findings suggest that while NLP methods show promise, no single approach is sufficient, and ethical considerations in automated keyword extraction are crucial for responsible stewardship.