Orchestra-o1: Omnimodal Agent Orchestration
Quick Answer
Orchestra-o1 is a novel omnimodal agent orchestration framework that enhances collaboration across text, image, audio, and video modalities.
Quick Take
Orchestra-o1 is a novel omnimodal agent orchestration framework that enhances collaboration across text, image, audio, and video modalities. It achieves a 10.3% accuracy improvement over the second-best approach on the OmniGAIA benchmark and introduces DA-GRPO for efficient training, setting a new state-of-the-art for open-source omnimodal agents.
Key Points
- Introduces a unified orchestration mechanism for multi-modal agent collaboration.
- Achieves 10.3% higher accuracy on OmniGAIA benchmark compared to the second-best method.
- Supports modality-aware task decomposition and parallel sub-task execution.
- Utilizes decision-aligned group relative policy optimization for training.
- Sets a new state-of-the-art for open-source omnimodal agents.
Paper Resources
Article Content
From source RSS / original summaryarXiv:2606. 13707v1 Announce Type: new Abstract: The recent success of agent swarms has shifted the paradigm of large language model (LLM)-based agents from single-agent workflows to multi-agent systems, highlighting the importance of agent orchestration for task decomposition and collaboration. However, existing orchestration frameworks are limited to a narrow set of modalities and struggle to generalize to more complex settings where heterogeneous modalities coexist and interact.
This limitation becomes particularly pronounced in omnimodal scenarios, where tasks require the unified understanding and coordination of diverse inputs such as text, image, audio, and video. In this work, we propose Orchestra-o1, an omnimodal agent orchestration framework designed to support efficient agent collaboration across multiple modalities.
Orchestra-o1 introduces a unified orchestration mechanism that enables modality-aware task decomposition, online sub-agent specialization, and parallel sub-task execution. This scalable design allows agent systems to effectively tackle complex real-world tasks involving heterogeneous information sources, surpassing the second-best approach by 10. 3% accuracy on the OmniGAIA benchmark.
Furthermore, we introduce decision-aligned group relative policy optimization (DA-GRPO), an efficient agentic reinforcement learning approach for training Orchestra-o1-8B, which also achieves state-of-the-art performance against all existing open-source omnimodal agents.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →Arbor: Tree Search as a Cognition Layer for Autonomous Agents
Arbor introduces a multi-agent framework utilizing structured tree search for optimizing LLM inference, achieving up to 193% throughput-latency improvement compared to vendor-optimized systems. It employs an Orchestrator and Critic agent for stability and coordination, demonstrating hardware-agnostic performance with minimal variance.