ARIA: Runtime Inspection and Amendment for Multi-Agent Systems
Quick Answer
The ARIA framework enables real-time intervention in multi-agent systems by learning from over 24,000 trajectories, including 3,953 verified repair pairs.
Quick Take
This allows for effective runtime corrections without the need for repeated testing, enhancing performance across various frameworks and models.
Key Points
- ARIA collects data from four frameworks and six benchmarks.
- The framework includes the first large-scale dataset of failed trajectories with verified repairs.
- It trains a compact model to recognize and correct ongoing execution errors.
- The ARIA model shows strong performance in both post-hoc and runtime settings.
- Code for the ARIA framework is publicly available for further research.
Paper Resources
📖 Reader Mode
~2 min readAbstract:In LLM-based multi-agent systems (MAS), a local mistake can propagate as other agents build on it, turning a correctable error into a failed task. Diagnosing completed runs or repeatedly re-executing them to search for corrections does not provide a learned policy for intervening while execution is still in progress. We present Agent Runtime Inspection and Amendment (ARIA), a framework for learning when and how to repair ongoing multi-agent executions. Our key intuition is to turn execution-verified repairs into reusable supervision, moving repair search into data construction and training so that a learned model can intervene at runtime without repeatedly testing alternatives. First, we collect over 24{,}000 successful and failed trajectories across four multi-agent frameworks, six benchmarks, and eight host models. From these trajectories, we construct the ARIA dataset, to our knowledge the first large-scale dataset pairing failed multi-agent trajectories with execution-verified repairs. It contains 3{,}953 verified repair pairs, each specifying what went wrong, where to intervene, and how execution should continue. Then, we develop the ARIA framework for MAS-agnostic runtime inspection and amendment, designed to integrate with any MAS that exposes execution trajectories and supports continuation replacement. Using verified repairs as training examples, we teach a compact model to recognize when intervention is needed and how to correct an ongoing run. The aim is to learn repair strategies that apply to related failures in new configurations, rather than merely reproduce individual corrections. Finally, the trained ARIA model demonstrates strong repair performance in both post-hoc and runtime settings, with generalization to unseen frameworks and host models. Our code is available at this https URL.
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.20780 [cs.AI] |
| (or arXiv:2610.20780v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20780 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Bolin Shen [view email]
[v1]
Wed, 14 Oct 2026 18:27:47 UTC (391 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →VEPA: Verified Evolving Formal Policy for Lifelong Agent Guardrails
VEPA introduces a framework for verified evolving formal policies for LLM agents, enhancing safety by adapting guardrails based on deployment feedback. It outperforms existing methods across benchmarks like ASSEBench-Safety and ASSEBench-Security, ensuring deterministic policy enforcement even under challenging conditions.