Occlusion-Aware Physics-Semantic Keyframe Selection for Robust Video Editing
Quick Take
The proposed occlusion-aware physics-semantic keyframe selection framework enhances video editing by automatically identifying optimal anchor frames, addressing issues of occlusion and motion. This method improves localization and consistency in edits without manual annotations, as demonstrated by extensive experiments on challenging benchmarks.
Key Points
- Framework selects optimal anchor frames to enhance video editing robustness.
- Addresses occlusion, viewpoint changes, and fast object motion issues.
- Utilizes structural completeness and cycle-consistent tracking for reliability.
- Generates dense spatiotemporal masks for improved editing supervision.
- Demonstrated effectiveness on challenging video editing benchmarks.
Article Content
From source RSS / original summaryarXiv:2605. 23192v1 Announce Type: new Abstract: Video editing has recently achieved remarkable progress with diffusion-based generative models, enabling diverse object-level manipulations from natural language instructions. However, existing methods often struggle under occlusion, viewpoint changes, and fast object motion, where unreliable visual observations lead to inaccurate localization, temporal flickering, and inconsistent edits.
In this work, we identify the absence of reliable visual anchors as a fundamental bottleneck in occlusion-robust video editing. To address this issue, we propose an occlusion-aware physics-semantic keyframe selection framework that automatically identifies an optimal anchor frame for downstream editing.
Specifically, our method evaluates candidate frames from three complementary perspectives: structural completeness for avoiding truncated observations, cycle-consistent tracking stability for measuring physical reliability, and vision-language-based attribute visibility for ensuring semantic clarity. The selected keyframe is then propagated through bidirectional tracking to generate dense spatiotemporal masks, which are used as auxiliary supervision for a diffusion-based video editing backbone.
By transforming occlusion handling from explicit reconstruction into reliable anchor selection, our framework enables precise and temporally consistent editing without requiring manual annotations. Extensive experiments on challenging video editing benchmarks demonstrate the effectiveness and high-quality performance of our method.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →Evi-Steer: Learning to Steer Biomedical Vision-Language Models through Efficient and Generalizable Evidential Tuning
Evi-Steer introduces a novel evidential tuning framework for BiomedCLIP, achieving 0.11% parameter updates while enhancing uncertainty-aware fine-tuning. It outperforms state-of-the-art methods across 15 biomedical imaging datasets, proving effective in few-shot learning and domain shifts for clinical applications.