LadderEdit: Edit-Level Residual Compression for Memory-Efficient Lifelong Editing of LLMs
Quick Answer
LadderEdit introduces a memory-efficient method for lifelong editing of LLMs by compressing LoRA adapters, achieving 5.2x less memory usage while maintaining effectiveness across 50,000 edits on models like LLaMA-3-8B and Mistral-7B.
Quick Take
This approach retains coverage by storing edits as low-rank sketches and promoting them only when necessary, ensuring compliance with rewrite contracts.
Key Points
- LadderEdit compresses LoRA adapters after acquisition for memory efficiency.
- Achieves 5.2x reduction in memory usage during lifelong editing.
- Maintains effectiveness across 50,000 sequential edits on multiple benchmarks.
- Edits are stored as low-rank sketches, promoting only when necessary.
- Ensures compliance with rewrite, generalization, and locality contracts.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Lifelong editing of LLMs requires storing thousands of edits after acquisition. A widely used family of approaches attaches one LoRA adapter per edit, which preserves behavior but grows linearly in storage. To address this challenge, we propose LadderEdit, a method that compresses each LoRA adapter after it is acquired. Each edit is first stored at low rank as a cheap sketch. We then check whether this sketch still satisfies the rewrite, generalization, and locality contract on probe prompts. Edits that pass keep the sketch; those that fail are promoted to a higher rank along a ladder until the contract is met. Because every edit retains some representation, coverage is maintained, and only hard edits consume more rank. Across ZsRE, CounterFact, and WikiBigEdit benchmarks on LLaMA-3-8B, Mistral-7B, and Qwen2.5-7B, LadderEdit tracks exact LoRA storage at 5.2x less memory and remains effective at 50,000 sequential edits.
| Comments: | EMNLP 2026 Main Conference Long Paper |
| Subjects: | Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Multimedia (cs.MM) |
| Cite as: | arXiv:2610.11160 [cs.CL] |
| (or arXiv:2610.11160v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.11160 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xiaofeng Liu [view email]
[v1]
Thu, 8 Oct 2026 03:20:35 UTC (531 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →The "10th Juror": Open-Set Standpoint Screening for Bureaucratic Bias Detection
MARS-Gov introduces a framework for detecting bureaucratic bias in Dutch government documents, achieving a new state-of-the-art F1 score of 0.880. This model outperforms existing zero-shot detectors by 20.2 points and reduces unnecessary interventions to just 2.5%. The framework's dynamic '10th juror' adapts to emerging biases, enhancing legal language processing.