Do as I Say, Not as I Do: Instruction-Induction Conflict in LLMs
Quick Answer
The study reveals that instruction-following in language models (LLMs) can conflict with hardcoded patterns, leading to variable compliance rates from 1% to 99% across 13 models.
Quick Take
The study reveals that instruction-following in language models (LLMs) can conflict with hardcoded patterns, leading to variable compliance rates from 1% to 99% across 13 models. Robustness is influenced by instruction content and output format, with multi-token responses being more resistant to induction pressure. Despite achieving 83.5% accuracy in self-prediction, models underestimate their susceptibility to pattern-following.
Key Points
- Instruction-following rates vary significantly from 1% to 99% across different models.
- Robustness is higher with multi-token outputs compared to single-token responses.
- Chain-of-thought reasoning enhances robustness but does not eliminate induction susceptibility.
- Models predict their behavior with 83.5% accuracy but underestimate their resistance to induction.
- Instruction content alignment with trained values increases resistance to pattern-following.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Language models are trained to follow instructions, but they are also powerful pattern completers. What happens when these two objectives conflict? We construct conversations in which a user instruction to behave in a target way T (e.g., always output a specific token, answer in a particular language, or adopt a persona) is opposed by N hardcoded assistant turns demonstrating a competing pattern P. We then measure instruction-following (IF) rates in this setting, across 13 models and 16 different instructions, for up to 50 turns. Average instruction-following rates range from 1% to 99% across models, largely uncorrelated with standard capability benchmarks. The transition from instruction-following to pattern-following is universal but highly model-dependent. Robustness is modulated both by instruction content, with models resisting induction longer when instructions align with their trained value priors, and by output format, with diverse multi-token responses proving substantially more resistant than single-token outputs. Chain-of-thought reasoning improves robustness but does not eliminate susceptibility, and can produce dissociation between correct deliberation and incorrect output. When asked to predict their behavior in this setting, models achieve 83.5% accuracy on average but systematically underestimate their own resistance to induction pressure. These results suggest that instruction-following remains brittle under induction pressure even for otherwise capable models, and that output diversity, rather than semantic engagement with the input, is the primary factor predicting robustness.
| Comments: | 31 pages |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.20382 [cs.CL] |
| (or arXiv:2605.20382v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2605.20382 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Carolina Camassa [view email]
[v1]
Tue, 19 May 2026 18:32:20 UTC (5,155 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Quantifying Prior Dominance in Systems
The study introduces the Normalized Context Utilization (NCU) metric to evaluate Retrieval-Augmented Generation (RAG) systems, revealing that Small Language Models (SLMs) outperform larger models in factual extraction. The findings indicate that traditional scaling laws yield diminishing returns, with a commercial API frequently failing against adversarial evidence due to systemic confidence collapse.