SERC: LDPC-Inspired Semantic Error Correction for Retrieval-Augmented Generation
Quick Take
The LDPC-inspired Semantic Error Correction (SERC) framework enhances retrieval-augmented generation by mitigating LLM hallucinations. Evaluated on LongForm Bio and TruthfulQA with Llama-3-8B and Qwen2.5-14B, SERC significantly improves factual precision and allows small language models to outperform larger ones, reducing verification overhead and achieving cost-effective fidelity.
Key Points
- SERC reformulates text generation as a semantic noisy channel for error correction.
- Utilizes sparse verification queries to efficiently validate generated responses.
- Outperforms intrinsic self-correction methods and strong baselines in factual precision.
- Enables small language models to exceed larger models in hallucination reduction.
- Achieves optimal cost-fidelity trade-off in resource-constrained environments.
Article Content
From source RSS / original summaryarXiv:2605. 28837v1 Announce Type: new Abstract: While Large Language Models (LLMs) have demonstrated remarkable capabilities, their reliability is significantly compromised by hallucinations. Existing intrinsic self-correction methods attempt to address this, but often fail due to self-bias, where models struggle to identify errors in their own outputs without external verification.
To overcome these limitations, we propose the LDPC-inspired semantic error correction for retrieval-augmented generation (SERC), providing a theoretical framework to interpret and mitigate LLM hallucinations. We reformulate the text generation process as a semantic noisy channel, treating generated responses as noise-corrupted codewords.
Inspired by low-density parity-check (LDPC) codes, SERC employs a sparse verification strategy: instead of exhaustively checking all facts, it generates low-density verification queries and validates them against external evidence to efficiently detect and correct errors. We evaluate SERC on LongForm Bio and TruthfulQA benchmarks using Llama-3-8B and Qwen2. 5-14B.
Experimental results demonstrate that SERC outperforms both intrinsic self-correction methods and strong retrieval-augmented baselines, demonstrating significant gains especially in factual precision (FactScore). Notably, SERC enables small language models (SLMs) to surpass the performance of larger baselines in hallucination reduction and information preservation.
Our findings demonstrate that SERC provides a training-free, model-agnostic solution that significantly reduces verification overhead compared to dense methods, achieving an optimal trade-off between cost and fidelity in resource-constrained environments.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?
The reliability of LLM judges for evaluating deep research agents is critically assessed using the REFLECT benchmark.