MechELK: A Mechanistic Interpretability Framework for Eliciting Latent Knowledge in Large Language Models
Quick Answer
MechELK is a novel three-stage framework that enhances the extraction of latent knowledge in large language models, achieving an average elicitation accuracy of 84.7% on benchmarks like TruthfulQA, outperforming existing methods by notable margins.
Quick Take
MechELK is a novel three-stage framework that enhances the extraction of latent knowledge in large language models, achieving an average elicitation accuracy of 84.7% on benchmarks like TruthfulQA, outperforming existing methods by notable margins. It effectively identifies hidden knowledge even when surface outputs are incorrect, proving valuable for AI safety applications.
Key Points
- MechELK integrates mechanistic interpretability with latent knowledge elicitation in LLMs.
- The framework consists of Locate, Verify, and Elicit stages for knowledge extraction.
- Achieved 84.7% elicitation accuracy, surpassing CCS by 6.2% and linear probing by 9.1%.
- Identifies latent knowledge in 78.3% of cases with incorrect or evasive outputs.
- Demonstrates potential for improving AI safety through deceptive alignment detection.
Paper Resources
Article Content
From source RSS / original summaryarXiv:2605. 28825v1 Announce Type: new Abstract: Large language models (LLMs) frequently encode factual and reasoning knowledge in their internal representations that is not faithfully reflected in their surface-level outputs -- a phenomenon known as \emph{latent knowledge}.
Existing approaches to eliciting latent knowledge, such as Contrastive Consistency Search (CCS), rely on contrastive activation patterns and struggle with complex multi-step reasoning tasks, while mechanistic interpretability tools have primarily been used to \emph{understand} model behavior rather than to \emph{extract} hidden knowledge. We present \textbf{MechELK}, a unified three-stage framework that bridges mechanistic interpretability and latent knowledge elicitation.
MechELK operates through: (1) \textbf{Locate} -- using Sparse Autoencoder (SAE) feature analysis and activation patching to identify knowledge-bearing representations; (2) \textbf{Verify} -- employing causal probing to distinguish genuine latent knowledge from spurious correlations; and (3) \textbf{Elicit} -- applying representation engineering to surface hidden knowledge without modifying model weights.
Evaluated on TruthfulQA, a curated Deceptive Alignment benchmark, and the Quirky LM dataset, MechELK achieves an average elicitation accuracy of 84. 7\%, outperforming CCS by 6. 2\% and direct linear probing by 9. 1\%. Crucially, MechELK successfully identifies latent knowledge in 78. 3\% of cases where the model's surface output is incorrect or evasive, demonstrating its utility for AI safety applications including deceptive alignment detection.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Letting the Data Speak: Extracting Keywords from Crowdsourced Collections with AI
The study evaluates three NLP approaches—Named Entity Recognition, Keyword Extraction, and Topic Modelling—using the Their Finest Hour Online Archive to automate keyword extraction from crowdsourced WWII collections. Findings suggest that while NLP methods show promise, no single approach is sufficient, and ethical considerations in automated keyword extraction are crucial for responsible stewardship.