Between Suppression and Collapse: Evaluating Narrative Unlearning with LENS
Quick Answer
The LENS protocol evaluates narrative unlearning in large language models (LLMs) like Lapa LLM and Qwen-14B, revealing that narrative suppression can be achieved without degrading output quality.
Quick Take
The introduction of the Suppression-Collapse Efficiency (SCE) score demonstrates effective checkpoint selection for reducing disinformation narratives, with findings indicating that suppression can extend beyond direct prompts.
Key Points
- LENS evaluates narrative suppression across multiple resistance levels in .
- Suppression-Collapse Efficiency (SCE) score rewards effective narrative suppression.
- Experiments involved four multilingual models: Lapa LLM, Gemma-12B, Qwen-14B, and TAIDE-Gemma.
- Results indicate that narrative suppression can transfer beyond direct forget prompts.
- Entity recovery can occur post-unlearning, revealing real-world actors linked to narratives.
DeepSignal Analysis
What happened
The LENS protocol evaluates narrative unlearning in large language models (LLMs) like Lapa LLM and Qwen-14B. It demonstrates that narrative suppression can occur without compromising output quality. The introduction of the Suppression-Collapse Efficiency (SCE) score aids in selecting effective checkpoints to mitigate disinformation narratives.
Key evidence
- LENS evaluates narrative suppression across various resistance levels, including direct and abstract prompts, using models like Lapa LLM and Qwen-14B.
- The SCE score is designed to reward effective narrative suppression while penalizing any degradation in output quality.
- Experiments show that narrative suppression can extend beyond direct prompts, indicating a broader capability for unlearning in LLMs.
Why it matters
Understanding how LLMs can suppress disinformation narratives is crucial for improving their reliability and ethical use. The findings suggest that narrative unlearning can be effectively measured and implemented, which is significant for applications in misinformation management and responsible AI deployment. The implications of this research could influence future developments in LLM training and evaluation.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Large language models (LLMs) can reproduce disinformation-aligned narrative frames as plausible explanations, raising the question of whether existing machine-unlearning algorithms can suppress this behavior. We introduce Level-based Evaluation of Narrative Suppression (LENS), a contextualization based evaluation protocol for testing target narrative reproduction across direct, attributed, contrastive, and abstract resistance levels. We evaluate two source-grounded narratives: one framing Russia's war against Ukraine as forced by NATO expansion, and one framing the United States as exploiting or abandoning Taiwan. The experiments cover four near-12B multilingual instruction models: Lapa LLM, Gemma-12B, Qwen-14B, and TAIDE-Gemma.
We introduce the Suppression-Collapse Efficiency (SCE) score as a checkpoint selection summary that rewards target-narrative suppression while penalizing degraded outputs. Our results shows that selected checkpoints can reduce narrative reproduction and suppression may transfer beyond direct forget prompts. We also report entity recovery as a separate side effect: abstract A/B/C prompts can cause models to recover the real-world actors associated with the target frame after unlearning. These findings demonstrate that LENS is a successful diagnostic protocol for both reporting and guiding the further study of the deeper structure of narrative unlearning.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.22657 [cs.CL] |
| (or arXiv:2607.22657v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.22657 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Viktoriia Makovska [view email]
[v1]
Sat, 27 Jun 2026 15:20:38 UTC (158 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.