Behavior Leverage Imbalance in Multi-Teacher On-Policy Distillation
Quick Answer
The paper introduces a multi-teacher on-policy distillation strategy that improves tool-call accuracy while reducing over-calling in agentic language models.
Quick Take
The paper introduces a multi-teacher on-policy distillation strategy that improves tool-call accuracy while reducing over-calling in agentic language models. By implementing Soft Clamp, a divergence calibration method, the model's over-calling rate decreased from 13.7% to 9.0% on APIGen-MT without sacrificing decision accuracy. This highlights the importance of monitoring teacher signal locations in training.
Key Points
- Soft Clamp reduces over-calling from 13.7% to 9.0% on APIGen-MT.
- The model maintains decision accuracy while improving tool-call recall.
- Behavior leverage imbalance affects global generation modes significantly.
- Local token-level signals control generation more than aggregate losses.
- Multi-teacher OPD should focus on where teacher signals are applied.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Agentic language models must learn when to call tools, when to consume tool responses, and when to answer directly. This makes multi-teacher on-policy distillation a natural training strategy: one teacher can specialize in tool calls, another in direct responses, and the student can learn from both on
its own generated distribution. We show that this strategy can induce a behavior shift that is invisible from aggregate losses alone. In a two-teacher tool-use setting, vanilla generalized knowledge distillation improves tool-call recall but also moves the model toward over-calling, where it calls tools
on examples that should be answered directly. Aggregate explanations are insufficient: tool-call samples do not receive more token exposure, and full-sequence per-token divergence is not larger for the tool-call teacher. We instead analyze behavior leverage imbalance: local token-level signals at mode-
entry and structural positions, such as <tool_call> and function names, can have disproportionate control over the global generation mode. We propose Soft Clamp, a per-token divergence calibration method that dynamically compresses extreme token-level Jensen-Shannon divergence while preserving nonzero
gradients. On APIGen-MT, Soft Clamp reduces over-calling from 13.7% to 9.0% relative to vanilla GKD while matching its decision accuracy. In a BFCL multi-turn diagnostic, it also lowers tool-call loops and repeated calls among GKD variants. These results suggest that multi-teacher OPD should monitor
where teacher signals act, not only how large they are in aggregate.
| Comments: | 17 pages including appendix, 6 figures |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2607.07050 [cs.CL] |
| (or arXiv:2607.07050v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.07050 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jiabin Shen [view email]
[v1]
Wed, 8 Jul 2026 06:26:13 UTC (135 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Quantifying Prior Dominance in Systems
The study introduces the Normalized Context Utilization (NCU) metric to evaluate Retrieval-Augmented Generation (RAG) systems, revealing that Small Language Models (SLMs) outperform larger models in factual extraction. The findings indicate that traditional scaling laws yield diminishing returns, with a commercial API frequently failing against adversarial evidence due to systemic confidence collapse.