Light-Omni: Reflex over Reasoning in Agentic Video Understanding with Long-Term Memory
Quick Answer
Light-Omni is a novel multimodal agent framework for video understanding that enhances performance by leveraging long-term memory, achieving a 2.4% accuracy gain over M3-Agent, a 12.1× speedup, and a 2.6× increase in GPU memory efficiency.
Quick Take
Light-Omni is a novel multimodal agent framework for video understanding that enhances performance by leveraging long-term memory, achieving a 2.4% accuracy gain over M3-Agent, a 12.1× speedup, and a 2.6× increase in GPU memory efficiency. It eliminates the need for heavy iterative reasoning by utilizing dual contextual states for reflexive responses and semantically aligned retrieval. Extensive experiments validate its effectiveness across multiple benchmarks.
Key Points
- Light-Omni uses dual contextual states for efficient video understanding.
- Achieves a 2.4% accuracy improvement over M3-Agent in benchmarks.
- Delivers a 12.1× speedup and 2.6× better GPU memory efficiency.
- Eliminates heavy reasoning by providing instant global context.
- Serves as a memory system to enhance existing MLLMs' performance.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Agentic video understanding equips models with long-term memory to autonomously process and respond to continuous, long-horizon multimodal streams. However, advanced video agents often rely on ``detective-style'' iterative reasoning for action control (e.g., $\mathtt{search}$) and evidence aggregation, incurring prohibitive costs and latency. We argue that such heavy reasoning primarily compensates for the lack of global context and semantic misalignment in retrieval. This paper introduces Light-Omni, a multimodal agent framework for reflexive and lightweight video understanding. It achieves this through dual contextual states that instantly build the required context in a single forward pass. First, we maintain a global state, a finite-sized multimodal script continuously consolidated from episodic memory, serving as the global context for Light-Omni. Through hierarchical merging, it preserves recent details while summarizing past events. Second, conditioned on this global context, we generate a parametric latent state that directly drives autonomous actions and produces retrieval embeddings, with minimal latency. Benefiting from this coupled design, Light-Omni achieves semantically aligned retrieval and reflexive responses while avoiding iterative reasoning. Extensive experiments validate the effectiveness of Light-Omni across multiple video benchmarks. Notably, it outperforms M3-Agent with an average 2.4% accuracy gain, a 12.1$\times$ speedup, and a 2.6$\times$ improvement in GPU memory efficiency. Furthermore, it serves as a memory system to enhance both the performance and efficiency of existing MLLMs. Project page: this https URL.
| Comments: | Project Page: this https URL |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2607.05511 [cs.CV] |
| (or arXiv:2607.05511v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2607.05511 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Chang Nie [view email]
[v1]
Mon, 6 Jul 2026 18:00:06 UTC (5,594 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →LLM-Guided ANN Index Optimization for Human-Object Interaction Retrieval
A phase-aware LLM agent optimizes human-object interaction retrieval, outperforming Optuna TPE by 33.3% and VDTuner by 34.2% on the HICO-DET benchmark. This method enhances throughput by 15.3x over UniIR and demonstrates strong transferability across vector database management systems.
