Visual Token Compression Enhances Robustness of MLLMs
Quick Answer
This study demonstrates that visual token pruning significantly improves the robustness of Multimodal Large Language Models (MLLMs) against jailbreak attacks and hallucinations, achieving an average 13.29% enhancement in defense.
Quick Take
By identifying and removing out-of-distribution visual tokens, the method not only strengthens model reliability but also reduces inference costs, validated across seven diverse benchmarks.
Key Points
- Average 13.29% improvement in defending against jailbreak attacks.
- Method effectively mitigates hallucinations in MLLMs.
- Visual tokens identified as out-of-distribution are iteratively pruned.
- Reduces inference costs while enhancing model robustness.
- Evaluated on seven diverse popular benchmarks.
DeepSignal Analysis
What happened
The study introduces a method for visual token pruning that enhances the robustness of Multimodal Large Language Models (MLLMs) against vulnerabilities like jailbreak attacks and hallucinations. This approach identifies and removes out-of-distribution visual tokens, leading to a reported average improvement of 13.29% in defense against such attacks.
Key evidence
- The method measures the distance between visual tokens and the language feature space to identify out-of-distribution tokens.
- The study evaluates the effectiveness of the pruning method across seven diverse benchmarks, demonstrating consistent performance improvements.
- The approach not only enhances model reliability but also reduces inference costs, providing a dual benefit.
Why it matters
Enhancing the robustness of MLLMs is critical as these models are increasingly deployed in real-world applications where vulnerabilities can lead to significant risks. By addressing issues like jailbreak attacks and hallucinations, this research contributes to the development of more reliable AI systems. The reduction of inference costs also makes these models more efficient, which is important for scalability and practical use.
Paper Resources
Source Excerpt
In this paper, we show for the first time that visual token pruning enhances the robustness of Multimodal (MLLMs), mitigating vulnerabilities such as jailbreak attacks and hallucinations. Given that vision and language modalities cannot be perfectly aligned, the misaligned visual tokens might act as out-of-distribution (OOD) inputs, leading to unpredictable outputs and introducing potential vulnerabilities. Building on this insight, we aim to enhance model robustness agains
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
ProMoE-FL introduces a Prototype-conditioned Mixture-of-Experts framework for multimodal federated learning, effectively addressing missing modalities. It outperforms existing methods on four chest X-ray datasets, demonstrating superior feature synthesis capabilities in both homogeneous and heterogeneous settings.