One Model to Loop Them All: Elastic Latent Recurrence for Language Models
Quick Answer
This paper shows that The Elastic Latent Recurrence (ELR) model enables efficient training of recurrent language models without the typical accuracy trade-off, achieving comparable performance to non-recurrent models.
Quick Take
ELR maintains one-loop accuracy while allowing additional loops to enhance performance, resulting in Koi-4B scoring 18 on the Artificial Analysis Intelligence Index at one loop and 29 at four loops. This innovation reduces inference costs and improves accuracy, particularly in MoE models.
Key Points
- ELR model achieves similar accuracy to non-recurrent models at one loop.
- Koi-4B scores 18 on the Artificial Analysis Intelligence Index at one loop.
- Four loops in ELR add 7.25 and 4.75 points to performance.
- MoE models benefit from 11.7% faster updates with reused expert routes.
- Inference cost remains comparable to non-recurrent models at one loop.
DeepSignal Analysis
What happened
The Elastic Latent Recurrence (ELR) model allows for efficient training of recurrent language models without sacrificing accuracy. It achieves comparable performance to non-recurrent models, with Koi-4B scoring 18 on the Artificial Analysis Intelligence Index at one loop and 29 at four loops.
Key evidence
- ELR maintains one-loop accuracy while allowing additional loops to enhance performance, achieving similar accuracy to non-recurrent models of the same size.
- In MoE models, reusing first-loop expert routes on later loops reduces mixed-depth update time by 11.7% without affecting accuracy.
- Koi-4B scores 0.52 points below and 1.04 points above its matched baseline on a ten-task average at one loop, with four loops adding 7.25 and 4.75 points.
Why it matters
The ELR model represents a significant advancement in recurrent language model training, as it eliminates the typical accuracy trade-off associated with such models. This innovation could lead to more efficient language processing applications, particularly in scenarios where inference costs are critical. The ability to adjust loop counts for improved accuracy without incurring additional costs could make ELR a preferred choice for developers.
Paper Resources
📖 Reader Mode
~2 min readAbstract:A latent recurrent language model applies a shared block of layers repeatedly, so one checkpoint can trade inference compute for accuracy by changing the loop count. At one loop, however, such models typically fall well short of a non-recurrent model of the same size, a one-loop tax that makes a separate non-recurrent model the better choice whenever latency matters. We present Elastic Latent Recurrence (ELR), a recipe for training looped language models without this tax. At one loop, an ELR model costs the same to serve as a non-recurrent model of the same size and, trained on the same data, reaches similar accuracy. Each additional loop then buys more accuracy for more inference compute. Three choices make this work: a gated state update with a learned step size, loop counts mixed within each optimizer update, and conditioning each pass on its loop index and budget. The first two preserve one-loop accuracy, and the third lowers loss at every additional loop. Mixed depths can be introduced late in pretraining to limit the added training cost, and with zero-initialized conditioning added at the switch, this late schedule reaches lower loss than mixed-depth training from scratch. In MoE models, reusing first-loop expert routes on later loops cuts mixed-depth update time by 11.7% with no measurable change in accuracy. At inference, keeping only the last loop's attention KV cache holds attention KV memory at its one-loop level while retaining most of the gain from additional loops. On 4B dense and 30B-A3B MoE hybrid models, ELR at one loop is 0.52 points below and 1.04 points above its matched baseline on a ten-task average, and four loops add 7.25 and 4.75 points. Carried through post-training, the recipe yields Koi-4B, a single checkpoint that scores an estimated 18 on the Artificial Analysis Intelligence Index at one loop, matching Qwen3.5-4B, and 29 at four loops.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.12582 [cs.CL] |
| (or arXiv:2610.12582v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.12582 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shantanu Acharya [view email]
[v1]
Thu, 8 Oct 2026 18:05:42 UTC (296 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →The "10th Juror": Open-Set Standpoint Screening for Bureaucratic Bias Detection
MARS-Gov introduces a framework for detecting bureaucratic bias in Dutch government documents, achieving a new state-of-the-art F1 score of 0.880. This model outperforms existing zero-shot detectors by 20.2 points and reduces unnecessary interventions to just 2.5%. The framework's dynamic '10th juror' adapts to emerging biases, enhancing legal language processing.