RAG-Stress: Probing the Limits of Evidence Reliance in Retrieval-Augmented Generation
Quick Answer
The RAG-Stress protocol reveals that reliance on retrieved evidence can mislead models, causing them to incorrectly replace accurate answers.
Quick Take
Evaluating fifteen systems, including API models and reinforcement learning agents, showed a misleading rate increase of 10.9 to 13.5 percentage points, emphasizing the need for careful assessment of evidence adherence versus factual reliability.
Key Points
- -Stress examines evidence reliance limits in retrieval-augmented generation.
- Misleading rate (MR) increased by 10.9 to 13.5 percentage points across three QA datasets.
- Fifteen systems evaluated include API models and reinforcement learning agents.
- Prioritizing documents leads to higher MR compared to relying on prior knowledge.
- Evidence adherence does not guarantee factual reliability in model responses.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Following retrieved evidence does not guarantee factual correctness: misleading evidence can induce a model to replace an answer it previously gave correctly. Standard accuracy measures obscure this behavior by combining answer replacement with preexisting errors. We introduce RAG-Stress, a controlled diagnostic protocol for examining the limits of evidence reliance in retrieval-augmented generation. The protocol holds the question and reference answer fixed, edits one assertion to support a designated incorrect answer, and crosses two source priority policies with three positions of the answer span within the evidence text. We measure misleading rate (MR) on each model's subset of questions answered correctly without retrieval, alongside clean accuracy on the full evaluation set. We evaluate fifteen systems spanning API models, open models, and search agents trained with reinforcement learning on TriviaQA-RC, HotpotQA, and SearchQA, with additional English and Chinese MedQA evaluations. Instructions that prioritize documents consistently produce higher MR than those permitting reliance on prior knowledge. Averaged over models and positions, the gap ranges from 10.9 to 13.5 percentage points across the three QA datasets. Mean MR follows End $>$ Beginning $>$ Middle under both policies, although individual models do not uniformly follow this ordering. A separate paired audit of 500 questions and two checkpoints supports increased harmful override without establishing a corresponding improvement in beneficial correction. These findings distinguish evidence adherence from factual reliability and motivate evaluating whether retrieved evidence preserves, replaces, or corrects a model's answers.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.11183 [cs.CL] |
| (or arXiv:2610.11183v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.11183 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Hao Chen [view email]
[v1]
Thu, 8 Oct 2026 03:40:48 UTC (3,009 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →The "10th Juror": Open-Set Standpoint Screening for Bureaucratic Bias Detection
MARS-Gov introduces a framework for detecting bureaucratic bias in Dutch government documents, achieving a new state-of-the-art F1 score of 0.880. This model outperforms existing zero-shot detectors by 20.2 points and reduces unnecessary interventions to just 2.5%. The framework's dynamic '10th juror' adapts to emerging biases, enhancing legal language processing.