What Intermediate Layers Know: Detecting Jailbreaks from Entropy Dynamics
Quick Answer
The study reveals that jailbreak vulnerabilities in LLMs like Llama, Qwen, and Gemma are primarily encoded in intermediate layers, with entropy dynamics providing a more informative signal than static statistics.
Quick Take
This indicates that harmful intent is better detected through evolving token-level entropy rather than final output assessments.
Key Points
- Intermediate layers show concentrated entropy dynamics relevant to jailbreak detection.
- Static prompt-level entropy statistics provide minimal discriminative power.
- Entropy evolution features outperform traditional metrics in identifying harmful intent.
- Findings apply across multiple models and adversarial benchmarks without extra training.
- Jailbreak behavior is reflected in structured uncertainty dynamics within the model.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Jailbreak attacks reveal a persistent weakness in aligned Large Language Models: carefully crafted prompts can elicit policy-violating responses despite safety training. While most defenses operate at the prompt or output level, it remains unclear how harmful intent is encoded within the model's internal representations. We investigate this question by analyzing token-level predictive entropy trajectories across layers of a frozen LLM using the logit lens. We find that static aggregate statistics of prompt-level entropy (e.g., mean, variance) carry little discriminative signal, whereas features capturing how entropy evolves across token positions, such as monotonic rank-based trend scores, are substantially more informative. Importantly, this signal is not uniform across model depth: it is concentrated in intermediate layers and degrades at the final layer, indicating that jailbreak-relevant structure is most pronounced in mid-network representations rather than at the output head. Across multiple models (Llama, Qwen, Gemma) and adversarial benchmarks, these entropy dynamics provide architecture-consistent separation without additional training. Together, our findings show that jailbreak behavior is reflected in structured intermediate uncertainty dynamics, clarifying both which entropy-derived features encode harmful intent and where in the network that signal is most pronounced.
| Comments: | Accepted at the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD) 2026. A short version accepted at EIML@ICML 2026 |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2606.25182 [cs.CL] |
| (or arXiv:2606.25182v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2606.25182 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Sofiia Nikolenko [view email]
[v1]
Tue, 23 Jun 2026 21:14:53 UTC (1,984 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.