MLUBench: A Benchmark for Lifelong Unlearning Evaluation in MLLMs
Quick Answer
MLUBench introduces a comprehensive benchmark for evaluating lifelong unlearning in multimodal large language models (MLLMs), revealing significant degradation in existing methods.
Quick Take
The proposed LUMoE method effectively mitigates this issue, preserving multimodal alignment while addressing unlearning challenges. The benchmark includes 127 entities across 9 classes and is open-sourced for further research.
Key Points
- MLUBench features 127 entities across 9 classes for lifelong unlearning evaluation.
- Existing unlearning methods show severe cumulative degradation in MLLMs.
- LUMoE effectively addresses degradation while maintaining multimodal alignment.
- The benchmark and source code are available on GitHub for public access.
- Lifelong unlearning poses unique challenges not seen in unimodal models.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Multimodal large language models (MLLMs) are trained on massive multimodal data, making data unlearning increasingly important as data owners may request the removal of specific content. In practice, these requests often arrive sequentially over time, giving rise to the challenging problem of MLLM Lifelong Unlearning. However, most existing benchmarks are limited in scale and scope, failing to capture the complexities of MLLM lifelong unlearning. To fill this gap, we introduce the MLUBench, a large-scale and comprehensive benchmark featuring 127 entities across 9 classes under lifelong unlearning requests. We perform extensive experiments using MLUBench and reveal that existing unlearning methods suffer from severe, cumulative degradation. More critically, we further identify the unique challenge of this problem: unlike in unimodal models, MLLM lifelong unlearning is constrained by the need to preserve multimodal alignment. Continually unlearning from one modality could degrade the entire model. To alleviate this challenge, we propose LUMoE, an effective method. Experiments demonstrate that LUMoE significantly mitigates the degradation problem faced by baselines. The source code and the MLUBench dataset are open-sourced in this https URL.
| Comments: | 36 pages, accepted to the ICML 2026 |
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2606.12809 [cs.AI] |
| (or arXiv:2606.12809v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2606.12809 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: He Li [view email]
[v1]
Thu, 11 Jun 2026 02:09:26 UTC (1,324 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →HOBA: Hierarchical On-Policy Bidding Agents for Adaptive Online Advertising
HOBA (Hierarchical On-policy Bidding Agents) is a novel hierarchical reinforcement learning framework that enhances online advertising bidding systems by improving adaptability and reducing hyperparameter tuning costs. It utilizes a for hyperparameter inference, a SARSA agent for expert model selection, and a dynamic expert pool for bid execution, achieving a +3.6% increase in target cost during large-scale deployment and outperforming state-of-the-art baselines on AuctionNet.