MLUBench: A Benchmark for Lifelong Unlearning Evaluation in MLLMs
Quick Answer
MLUBench introduces a comprehensive benchmark for evaluating lifelong unlearning in multimodal large language models (MLLMs), revealing significant degradation in existing methods.
Quick Take
The proposed LUMoE method effectively mitigates this issue, preserving multimodal alignment while addressing unlearning challenges. The benchmark includes 127 entities across 9 classes and is open-sourced for further research.
Key Points
- MLUBench features 127 entities across 9 classes for lifelong unlearning evaluation.
- Existing unlearning methods show severe cumulative degradation in MLLMs.
- LUMoE effectively addresses degradation while maintaining multimodal alignment.
- The benchmark and source code are available on GitHub for public access.
- Lifelong unlearning poses unique challenges not seen in unimodal models.
Paper Resources
Source Excerpt
Multimodal (MLLMs) are trained on massive multimodal data, making data unlearning increasingly important as data owners may request the removal of specific content. In practice, these requests often arrive sequentially over time, giving rise to the challenging problem of MLLM Lifelong Unlearning. However, most existing benchmarks are limited in scale and scope, failing to capture the complexities of MLLM lifelong unlearning. To fill this gap, we introduce the MLUBench, a la
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →AINTMA: Agentic AI Architecture for Autonomous Test Management with Generative Intelligence, Secure Cloud Communication and Adaptive Quality Analytics
AINTMA, an autonomous test management architecture utilizing six specialized AI agents, achieves 88.4% test prioritization accuracy and reduces defect escape rates from 8.3% to 2.1%. The system demonstrates a 340% ROI within nine months, showcasing the potential of agentic AI in enhancing software quality management in cloud environments.