Test, then Route: How Language Models Execute In-Context Conditional Rules Across Models and Languages
Quick Answer
This study investigates how language models, specifically Gemma-3-4B and Qwen, execute in-context conditional rules, revealing that while testing is modular, routing is not.
Quick Take
The findings show that a mid-stack residual band carries the predicate's truth value, with significant localization across multiple languages and models, indicating a complex interplay in rule execution.
Key Points
- Mid-stack residual band carries the predicate's truth value across models and languages.
- Patching reroutes answers with near-perfect predicate-outcome flips in 17 out of 18 cases.
- Routing direction is token-bound and non-transferable, indicating specificity to trained pairs.
- Gemma-3-4B shows high transferability (0.98) for the same pair across languages.
- Testing is modular, while routing exhibits a non-abstract, pair-specific behavior.
DeepSignal Analysis
What happened
The study examines how language models like Gemma-3-4B and Qwen handle in-context conditional rules. It finds that while the testing process is modular, the routing mechanism is not, indicating a complex interaction in rule execution across different models and languages.
Key evidence
- The research involved three open models from two families and six languages, using a fixed item bank to assess rule execution.
- A mid-stack residual band was identified as carrying the predicate's truth value, with significant localization across five predicate families.
- In Gemma-3-4B, the routing mechanism showed a high transfer rate of approximately 0.98 to the same pair in other languages, contrasting with Qwen's pair-specific routing.
Why it matters
Understanding how language models execute conditional rules is crucial for improving their design and functionality. The findings suggest that while testing can be modular, routing may require more nuanced approaches, impacting how these models are applied in real-world scenarios. This has implications for multilingual applications and the development of more effective AI systems.
What to watch
Paper Resources
📖 Reader Mode
~2 min readAbstract:When a language model follows an in-context conditional rule such as "if P(x) then A else B," does it assemble a runtime circuit with one module that tests the predicate and another that routes the answer? We probe this with activation patching under a four-donor design whose two swapped-rule donors make the condition and the answer word disagree, so each layer reveals which of the two it carries. Across three open models from two families and six languages sharing one fixed item bank, a mid-stack residual band carries the predicate's truth value: patching it reroutes the answer with predicate-outcome flip near 1.0 and mapping flip near 0.0, meeting a strict pre-specified isolation criterion in 17 of 18 cells, and the same localization holds across five predicate families. The router shows the opposite profile. A learned subspace flips A and B near-perfectly within the trained pair yet transfers to a new pair at approximately 0 in every model, while in Gemma-3-4B (the only model probed cross-lingually) it transfers at approximately 0.98 to the same pair in other languages. Under every probe we ran, the router direction is token-bound and non-transferable (largely answer-readout in Gemma, pair-specific in Qwen) rather than an abstract routing module. Test is modular; under these probes, route is not.
| Comments: | 19 pages, 16 figures. Code and data: this https URL |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2608.04183 [cs.CL] |
| (or arXiv:2608.04183v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2608.04183 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Luxshan Thavarasa [view email]
[v1]
Tue, 4 Aug 2026 19:40:30 UTC (331 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.