Learn-by-Wire Training Control Governance: Bounded Autonomous Training Under Stress for Stability and Efficiency
Quick Answer
This paper shows that The Learn-by-Wire Guard (LBW-Guard) enhances stability in LLM training by providing a governance layer above AdamW, achieving an 18.7% reduction in perplexity and a 1.10x speedup in training time.
Quick Take
The Learn-by-Wire Guard (LBW-Guard) enhances stability in LLM training by providing a governance layer above AdamW, achieving an 18.7% reduction in perplexity and a 1.10x speedup in training time. In stress tests, LBW-Guard maintained trainability at lower learning rates, outperforming AdamW significantly, which degraded under similar conditions.
Key Points
- LBW-Guard reduces final perplexity from 13.21 to 10.74 in Qwen2.5-7B training.
- Achieves a 1.10x speedup in training time from 392.54s to 357.02s.
- Remains trainable at final perplexity of 11.57 and 10.33 under high learning-rate stress.
- Demonstrates that stability-sensitive LLM training benefits from governance above optimizers.
- Gradient-clipping baselines do not replicate LBW-Guard's stability effects.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Modern language-model training is increasingly exposed to instability, degraded runs, and wasted compute, especially under aggressive learning-rate, scale, and runtime-stress conditions. This paper introduces Learn-by-Wire Guard (LBW-Guard), a bounded autonomous training-control governance layer that operates above AdamW. Rather than replacing the optimizer update rule, LBW-Guard observes training telemetry, interprets instability-sensitive regimes, and applies bounded control to optimizer execution while preserving fixed training objectives.
We evaluate LBW-Guard in a Qwen2.5-centered stress-and-robustness suite using WikiText-103, with Qwen2.5-7B as the empirical anchor, model-size comparisons against Qwen2.5-3B and Qwen2.5-14B, learning-rate stress tests, gradient-clipping baselines, and a no-LoRA TinyLlama-1B full-parameter sanity check. In the 7B reference setting, LBW-Guard reduces final perplexity from 13.21 to 10.74, an 18.7% improvement, while reducing end-to-end time from 392.54s to 357.02s, a 1.10x speedup. Under stronger learning-rate stress, AdamW degrades to 1885.24 final perplexity at LR=3e-3 and 659.76 at LR=1e-3, whereas LBW-Guard remains trainable at 11.57 and 10.33, respectively. Gradient-clipping baselines do not reproduce this effect.
These results support a scoped systems conclusion that stability-sensitive LLM training can benefit from a governance plane above the optimizer. LBW-Guard provides evidence that bounded runtime control can preserve productive compute under stress while remaining distinct from optimizer replacement and local gradient suppression.
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.19008 [cs.AI] |
| (or arXiv:2605.19008v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2605.19008 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Anis Radianis [view email]
[v1]
Mon, 18 May 2026 18:32:25 UTC (24 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →Information Limits and Attractor Dynamics in Economies of Frontier LLM Agents: A Pre-Registered Test
A pre-registered experiment on Claude Opus 4.8 investigates wealth growth and population misalignment in economies, revealing that relative growth aligns with claimed information but fails to demonstrate expected noise-maintained dispersion. The experiment cost $138.76 and is fully reproducible from cached outputs.