PCA: Persistence-Aware Compression and Aggregation for Fast Video Large Language Models
Quick Answer
The PCA method enhances Video Large Language Models (VLLMs) by introducing persistence-aware compression and aggregation, achieving a 1.8x to 2.5x speedup while maintaining high fidelity in visual information.
Quick Take
This training-free approach includes dynamic downsampling and motion enhancement modules, significantly improving efficiency and accuracy over existing methods.
Key Points
- PCA reduces computation for long-context modeling in VLLMs.
- Dynamic Downsampling removes redundant frames based on similarity.
- Persistence-Aware Motion Enhancement enriches keyframes with temporal context.
- PCA outperforms state-of-the-art methods in efficiency and accuracy.
- Code for PCA is open-sourced for public access.
DeepSignal Analysis
What happened
The PCA method introduces a training-free approach to enhance Video Large Language Models (VLLMs) by implementing persistence-aware compression and aggregation techniques. It achieves a speedup of 1.8x to 2.5x while preserving visual fidelity, utilizing dynamic downsampling and motion enhancement modules.
Key evidence
- PCA is designed to preserve high-fidelity raw visual information before the encoding stage, addressing redundancy in long-duration frames.
- The method includes a Dynamic Downsampling module that adaptively removes redundant frames based on frame-wise similarity analysis.
- Extensive experiments show that PCA consistently outperforms existing state-of-the-art approaches in efficiency and accuracy.
Why it matters
This advancement is significant for video processing as it addresses the inefficiencies caused by redundant frames in long-duration videos. By improving the speed and accuracy of VLLMs, PCA could enhance applications in video understanding, potentially impacting various fields such as entertainment, surveillance, and autonomous systems.
What to watch
Paper Resources
📖 Reader Mode
~2 min readAbstract:Despite advances in Video Large Language Models (VLLMs) that have displayed promising outcomes in video understanding, the redundancy in the long-duration frames remains a hindrance to efficient reasoning. This paper introduces a training-free $\mathbf{P}$ersistence-Aware $\mathbf{C}$ompression and $\mathbf{A}$ggregation (PCA) method designed to preserve high-fidelity raw visual information before the encoding stage. PCA can be built on arbitrary VLLMs and consists of two modules: 1) A Dynamic Downsampling (DD) module that adaptively removes redundant frames by analyzing frame-wise similarity. 2) A Persistence-Aware Motion Enhancement (PAME) module that enriches each selected keyframe by aggregating the temporal context of its neighbors, ensuring that essential information is preserved even after aggressive frame reduction. Our approach substantially reduces the computation of long-context modeling, while enhancing the performance of the baseline model. Extensive experiments demonstrate that PCA consistently outperforms existing state-of-the-art approaches in both efficiency and accuracy, achieving a speedup of 1.8$\times$ to 2.5$\times$ compared to the baseline VLLM. The code is open-sourced at this https URL.
| Comments: | Accepted by ACM MM 2026 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2607.22726 [cs.CV] |
| (or arXiv:2607.22726v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2607.22726 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zihan Song Mr. [view email]
[v1]
Wed, 22 Jul 2026 05:17:38 UTC (1,957 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
ProMoE-FL introduces a Prototype-conditioned Mixture-of-Experts framework for multimodal federated learning, effectively addressing missing modalities. It outperforms existing methods on four chest X-ray datasets, demonstrating superior feature synthesis capabilities in both homogeneous and heterogeneous settings.