MechELK: A Mechanistic Interpretability Framework for Eliciting Latent Knowledge in Large Language Models
Quick Take
MechELK is a novel three-stage framework that enhances the extraction of latent knowledge in large language models, achieving an average elicitation accuracy of 84.7% on benchmarks like TruthfulQA, outperforming existing methods by notable margins. It effectively identifies hidden knowledge even when surface outputs are incorrect, proving valuable for AI safety applications.
Key Points
- MechELK integrates mechanistic interpretability with latent knowledge elicitation in LLMs.
- The framework consists of Locate, Verify, and Elicit stages for knowledge extraction.
- Achieved 84.7% elicitation accuracy, surpassing CCS by 6.2% and linear probing by 9.1%.
- Identifies latent knowledge in 78.3% of cases with incorrect or evasive outputs.
- Demonstrates potential for improving AI safety through deceptive alignment detection.
Article Content
From source RSS / original summaryarXiv:2605. 28825v1 Announce Type: new Abstract: Large language models (LLMs) frequently encode factual and reasoning knowledge in their internal representations that is not faithfully reflected in their surface-level outputs -- a phenomenon known as \emph{latent knowledge}.
Existing approaches to eliciting latent knowledge, such as Contrastive Consistency Search (CCS), rely on contrastive activation patterns and struggle with complex multi-step reasoning tasks, while mechanistic interpretability tools have primarily been used to \emph{understand} model behavior rather than to \emph{extract} hidden knowledge. We present \textbf{MechELK}, a unified three-stage framework that bridges mechanistic interpretability and latent knowledge elicitation.
MechELK operates through: (1) \textbf{Locate} -- using Sparse Autoencoder (SAE) feature analysis and activation patching to identify knowledge-bearing representations; (2) \textbf{Verify} -- employing causal probing to distinguish genuine latent knowledge from spurious correlations; and (3) \textbf{Elicit} -- applying representation engineering to surface hidden knowledge without modifying model weights.
Evaluated on TruthfulQA, a curated Deceptive Alignment benchmark, and the Quirky LM dataset, MechELK achieves an average elicitation accuracy of 84. 7\%, outperforming CCS by 6. 2\% and direct linear probing by 9. 1\%. Crucially, MechELK successfully identifies latent knowledge in 78. 3\% of cases where the model's surface output is incorrect or evasive, demonstrating its utility for AI safety applications including deceptive alignment detection.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?
The reliability of LLM judges for evaluating deep research agents is critically assessed using the REFLECT benchmark.