Rethinking the Readout: Unlocking Video Backbones for AI-Generated Video Detection
Quick Answer
The study introduces Velocity Gated Patch Velocity Profiling (V-PVP), a novel readout method that enhances AI-generated video detection by leveraging video pretrained backbones.
Quick Take
By replacing the aggregation layer, V-PVP achieves a 95.28 AUC on AIGVDBench while maintaining the backbone's frozen state, demonstrating improved performance over traditional methods.
Key Points
- V-PVP replaces the aggregation layer with two parallel streams, adding only 0.5M parameters.
- Video backbones typically compress frames into global descriptors, losing temporal dynamics.
- The method improves performance across various video backbones in fine-tuning and probing.
- V-PVP reactivates the temporal potential of frozen video backbones for better AIGV detection.
- Code for V-PVP is publicly available for further research and implementation.
DeepSignal Analysis
What happened
The study presents a new method called Velocity Gated Patch Velocity Profiling (V-PVP) aimed at improving the detection of AI-generated videos. By modifying the aggregation layer of video pretrained backbones, V-PVP achieves a 95.28 AUC on the AIGVDBench benchmark while keeping the backbone parameters frozen.
Key evidence
- V-PVP replaces the aggregation layer with two parallel streams over the patch velocity field, adding approximately 0.5 million trainable parameters.
- Traditional video backbones often compress frames into a single global descriptor, which can hinder performance on AIGV benchmarks.
- The proposed method demonstrates consistent performance improvements across various video backbones in both end-to-end fine-tuning and linear probing settings.
Why it matters
This research highlights the limitations of current video detection methods that rely on global descriptors, which may overlook critical temporal dynamics. By reactivating the temporal potential of video backbones, V-PVP could enhance the accuracy of AI-generated video detection, a growing concern in media authenticity.
What to watch
Paper Resources
Source Excerpt
AI-generated videos (AIGVs) typically contain subtle temporal artifacts that arise from inter-frame inconsistencies rather than within individual frames. A detector that captures such artifacts should therefore benefit from video pretrained backbones over image only ones. In practice, however, video backbones with standard global readouts often fail to outperform strong image pretrained probes on AIGV benchmarks. We attribute this gap to excessive spatiotemporal aggregation in the readout. Video
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
ProMoE-FL introduces a Prototype-conditioned Mixture-of-Experts framework for multimodal federated learning, effectively addressing missing modalities. It outperforms existing methods on four chest X-ray datasets, demonstrating superior feature synthesis capabilities in both homogeneous and heterogeneous settings.