Eliciting Intrinsic Hallucinations in LLMs via Semantically Equivalent Adversarial Attacks
Quick Answer
This study introduces a framework to evaluate the robustness of large language models (LLMs) like GPT-5-mini against intrinsic hallucinations through adversarial attacks.
Quick Take
The findings reveal that even state-of-the-art models can experience a 50% degradation in contextual faithfulness when subjected to semantically equivalent perturbations, highlighting the fragility of in-context evidence utilization.
Key Points
- Proposes a novel framework for assessing robustness against intrinsic hallucinations.
- Evaluates 5 open-source and 5 closed-source models across 3 datasets.
- Demonstrates up to 50% degradation in contextual faithfulness for models like GPT-5-mini.
- Highlights the fragility of in-context evidence even in state-of-the-art models.
- Code for the framework is publicly available.
DeepSignal Analysis
What happened
A study evaluated the robustness of large language models (LLMs) against intrinsic hallucinations using adversarial attacks. The research found that even advanced models like GPT-5-mini can suffer a 50% reduction in contextual accuracy when faced with semantically equivalent perturbations, indicating a significant vulnerability in their performance.
Key evidence
- The study tested 5 open-source and 5 closed-source generator models across 3 datasets to assess their susceptibility to adversarial attacks.
- The proposed framework uses natural, semantically equivalent variations of user queries to stress-test models for intrinsic hallucinations.
- Findings showed that state-of-the-art models can experience up to a 50% degradation in contextual faithfulness due to meaning-preserving perturbations.
Why it matters
The findings underscore the fragility of LLMs in maintaining contextual integrity when faced with adversarial inputs. This raises concerns about the reliability of these models in real-world applications, where accurate information generation is critical. The results suggest a need for improved architectures and training methods to enhance grounding and reduce hallucinations.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Large language models (LLMs) are often used in conjunction with external knowledge sources to improve their factual accuracy and decrease hallucinations, through methods such as Retrieval-Augmented Generation (RAG). However, these systems remain susceptible to intrinsic hallucinations, where the model generates unfaithful or fabricated information that is not supported by the retrieved evidence. We propose a novel framework to assess model robustness against this phenomenon by stress-testing using natural, semantically equivalent variations of a user query found via adversarial optimization methods. We apply our framework, which enforces strict semantic equivalence constraints and an intrinsic hallucination objective, to a range of adversarial attack techniques across white-box, gray-box, and black-box adversarial settings. Evaluating these attacks on 5 open-source and 5 closed-source generator models across 3 datasets, we demonstrate that even state-of-the-art models are highly susceptible to meaning-preserving perturbations, which significantly degrade contextual faithfulness (by up to 50% for GPT-5-mini). Our findings indicate that faithful use of in-context evidence remains fragile even in state-of-the-art LLMs, motivating architectures and training objectives that enforce robust grounding independent of surface query form. Code is available at: this https URL
| Comments: | To be presented at COLM 2026 |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2608.04286 [cs.CL] |
| (or arXiv:2608.04286v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2608.04286 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Atri Sharma [view email]
[v1]
Tue, 4 Aug 2026 23:25:40 UTC (820 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.