Evidence Graph Consistency in Retrieval-Augmented Generation: A Model-Dependent Analysis of Hallucination Detection
Quick Answer
This paper shows that The Evidence Graph Consistency (EGC) framework identifies hallucination patterns in Retrieval-Augmented Generation (RAG) models, revealing a model-family split: while Llama-2 shows expected graph consistency with hallucinations, GPT-4, GPT-3.5, and Mistral-7B exhibit a reversal, indicating distinct hallucination characteristics across models.
Quick Take
This suggests that embedding-based consistency cannot serve as a universal detection method.
Key Points
- EGC computes five structural consistency measures as hallucination indicators.
- Evaluated on RAGTruth with 5,767 responses across six .
- Llama-2 models show expected graph consistency; others do not.
- Reversal in GPT-4 and GPT-3.5 indicates different hallucination patterns.
- Embedding-based graph consistency is not a model-independent detection signal.
Paper Resources
Source Excerpt
(RAG) reduces but does not eliminate hallucination in . Existing detection methods rely on flat similarity between generated answers and retrieved passages, ignoring structural relationships among evidence pieces and answer claims. We propose Evidence Graph Consistency (EGC), a framework that constructs a local evidence graph per response and computes five structural consistency measures as hallucination indicators. Evaluated on the full questi
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.