CORTEX: Token-Level Hallucination Detection in RAG via Comparative Internal Representations
Quick Answer
CORTEX is a token-level hallucination detection method for Retrieval-Augmented Generation (RAG) that improves localization of ungrounded content by comparing internal representations of LLMs with and without retrieved documents.
Quick Take
Experiments on two benchmarks demonstrate substantial performance gains in detecting hallucinations, reducing false positives and enhancing span consistency.
Key Points
- CORTEX identifies hallucinations at the token level for improved localization.
- It compares internal representations with and without retrieved documents.
- The method reduces false positives by leveraging document-grounded information.
- Post-processing smoothing enhances span-consistent predictions.
- Experiments show significant performance improvements across multiple benchmarks.
Paper Resources
📖 Reader Mode
~2 min readAbstract:In this paper, we propose CORTEX, a token-level hallucination detection method for Retrieval-Augmented Generation (RAG). In long-form RAG outputs, hallucinations often arise in localized spans rather than throughout an entire response. CORTEX therefore identifies ungrounded content at the token level, enabling fine-grained localization of hallucinations. The key intuition behind CORTEX is that tokens grounded in retrieved documents should be more strongly influenced by those documents than hallucinated tokens. To capture this document-induced effect, CORTEX compares internal representations of a large language model (LLM) under two conditions: with and without the retrieved documents. Instead of relying solely on each token's immediate sensitivity to the retrieved documents, CORTEX also leverages the propagation of document-grounded information through preceding tokens, reducing false positives for tokens whose evidence has already been absorbed into the context. Finally, CORTEX applies post-processing smoothing step that models the tendency of hallucination labels to persist over contiguous spans, reducing local noise and encouraging span-consistent predictions. Experiments on two RAG benchmarks and three LLMs show that CORTEX substantially improves token-level hallucination detection, with each component consistently contributing to performance gains.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2606.31033 [cs.CL] |
| (or arXiv:2606.31033v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2606.31033 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Kazuaki Furumai [view email]
[v1]
Tue, 30 Jun 2026 02:04:24 UTC (30,933 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.