DELTAVID: Enhancing Fine-Grained Spatiotemporal Perception with Cross-Video Differences
Quick Answer
DELTAVID introduces a framework for enhancing fine-grained spatiotemporal perception in video understanding by leveraging cross-video differences.
Quick Take
It includes DELTAVID-10K and DELTAVID-Bench for training and testing, significantly improving performance on various benchmarks like MMVU and VideoHolmes, and enabling better local evidence reasoning.
Key Points
- DELTAVID enhances local spatiotemporal perception in video multimodal language models.
- Introduces DELTAVID-10K and DELTAVID-Bench for scalable training and evaluation.
- Significantly improves performance on benchmarks like MMVU and VideoHolmes.
- Cross-video differences serve as effective supervision for fine-grained perception.
- Enables better reasoning of local evidence in general video understanding.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Video multimodal large language models have made strong progress on open-ended video understanding, but they still lack precise local spatiotemporal perception. When two videos share almost the same global semantics and differ only in a short time span or a small region, current models often fail to find the change and provide reliable evidence. We propose DELTAVID, a verifiable proxy-task framework that enhances fine-grained spatiotemporal perception with cross-video differences. The key idea is to turn cross-video spot-the-difference into a trainable perception signal, where a model identifies local changes, judges temporal boundaries, and organizes spatial evidence by comparing similar videos. To make this signal scalable to train and reliable to evaluate, we further introduce DELTAVID-10K and DELTAVID-Bench, which convert controllable local differences in real videos into evidence-labeled training and test samples. Experiments show that DELTAVID substantially improves performance on cross-video difference understanding and transfers the learned local evidence ability to general video understanding benchmarks, including MMVU, MLVU, Video-MME, VideoHolmes, VideoMMMU, LVBench, TempCompass, and LongVideoBench. These results show that cross-video differences are not only an effective way to diagnose fine-grained perception failures, but also a scalable proxy supervision that moves Video MLLMs from coarse semantic understanding toward fine-grained spatiotemporal evidence reasoning.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.02551 [cs.CV] |
| (or arXiv:2607.02551v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2607.02551 arXiv-issued DOI via DataCite |
Submission history
From: Yankai Yang [view email]
[v1]
Fri, 26 Jun 2026 16:05:57 UTC (17,660 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
ProMoE-FL introduces a Prototype-conditioned Mixture-of-Experts framework for multimodal federated learning, effectively addressing missing modalities. It outperforms existing methods on four chest X-ray datasets, demonstrating superior feature synthesis capabilities in both homogeneous and heterogeneous settings.