ViSAGE: Constructing Self-Correcting Memories for Long-Form Video Understanding
Quick Answer
ViSAGE is a multimodal memory framework that enhances long-form video understanding by constructing self-correcting, entity-centric memories.
Quick Take
It improves accuracy by 5.9% over existing methods through cross-modal binding and bidirectional memory refinement, addressing entity confusion and error propagation. The framework also incorporates cross-verification for better evidence alignment.
Key Points
- ViSAGE anchors entity identity through cross-modal binding over long temporal ranges.
- Bidirectional memory refinement propagates delayed identity evidence for improved reasoning.
- The framework enables abstention from unsupported answers when evidence is missing.
- ViSAGE outperforms the strongest baseline with a 5.9% accuracy increase.
- Multi-agent cross-verification assesses evidence under identity-evidence alignment constraints.
DeepSignal Analysis
What happened
ViSAGE is a new multimodal memory framework designed to enhance long-form video understanding. It constructs self-correcting, entity-centric memories, improving accuracy by 5.9% over existing methods. The framework employs cross-modal binding and bidirectional memory refinement to address issues like entity confusion and error propagation.
Key evidence
- ViSAGE improves accuracy by 5.9% compared to existing methods, demonstrating its effectiveness in long-form video understanding.
- The framework utilizes cross-modal binding to anchor entity identity over long temporal ranges, which helps maintain consistency.
- Multi-agent cross-verification is incorporated to assess retrieved evidence, allowing for abstention when evidence is missing.
Why it matters
The development of ViSAGE addresses significant challenges in long-form video understanding, particularly the issues of entity confusion and error propagation. By enhancing memory accuracy and consistency, it has the potential to improve applications in various fields, including AI-driven video analysis and automated content generation. This could lead to more reliable AI systems capable of understanding complex multimedia content.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Multimodal agents operating in long-horizon environments must build and continually update multimedia memories to support entity-consistent, temporally grounded reasoning. However, existing agentic memory approaches often discard fine-grained dentity cues under aggressive compression and segment-wise processing. They also rely heavily on vector similarity retrieval, which can surface semantically related yet identity-mismatched evidence, leading to entity confusion, error propagation, and hallucinated answers.
We propose ViSAGE, a multimodal agentic memory framework that constructs self-correcting, entity-centric memories. Specifically, ViSAGE anchors entity identity via cross-modal binding over long temporal ranges. It then applies bidirectional memory refinement to propagate delayed identity evidence, retroactively unifying historical records and improving future reasoning. We also introduce multi-agent cross-verification to assess retrieved evidence under an identity-evidence alignment onstraint, enabling abstention instead of unsupported answers when evidence is missing. Extensive results demonstrate that ViSAGE consistently outperforms the strongest baseline, achieving 5.9% higher accuracy.
| Comments: | Accept by ACMMM 2026 |
| Subjects: | Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2607.28678 [cs.AI] |
| (or arXiv:2607.28678v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2607.28678 arXiv-issued DOI via DataCite (pending registration) |
|
| Related DOI: | https://doi.org/10.1145/3767308.3835852
DOI(s) linking to related resources |
Submission history
From: Yifan Zhang [view email]
[v1]
Wed, 29 Jul 2026 10:25:23 UTC (942 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →HOBA: Hierarchical On-Policy Bidding Agents for Adaptive Online Advertising
HOBA (Hierarchical On-policy Bidding Agents) is a novel hierarchical reinforcement learning framework that enhances online advertising bidding systems by improving adaptability and reducing hyperparameter tuning costs. It utilizes a for hyperparameter inference, a SARSA agent for expert model selection, and a dynamic expert pool for bid execution, achieving a +3.6% increase in target cost during large-scale deployment and outperforming state-of-the-art baselines on AuctionNet.