Toward Reliable Design of LLM-Enabled Agentic Workflows: Optimizing Latency-Reliability-Cost Tradeoffs
Quick Answer
This paper presents a framework for optimizing latency, reliability, and cost in LLM-enabled workflows, introducing performance models for both LLM and non-LLM agents.
Quick Take
Key findings include a water-filling token allocation policy and optimal workflow reliability characterized by shadow prices, which can significantly impact the design of sequential workflows under given constraints.
Key Points
- Introduces performance models for and non-LLM agents to analyze tradeoffs.
- Develops a water-filling token allocation policy for optimal resource distribution.
- Characterizes optimal workflow reliability using shadow prices for better decision-making.
- Focuses on sequential workflows under latency and cost constraints.
- Impacts the design of AI systems that integrate multiple interacting agents.
Paper Resources
Article Excerpt
From source RSS / original summaryarXiv:2605. 23929v1 Announce Type: new Abstract: Modern AI systems increasingly rely on workflows composed of multiple interacting agents, some powered by (LLMs) and others by conventional computational modules. This paper analyzes the fundamental tradeoffs between latency, reliability, and cost in LLM-enabled agentic workflows.
We introduce performance models for both LLM and non-LLM agents that capture the relationship between computational effort and output quality, incorporating the impact of reasoning and output tokens for LLM agents using a parametric exponential reliability function. Then, we study the design of sequential workflows under latency and cost constraints. Main results include a water-filling token allocation policy and characterizations of optimal workflow reliability in terms of shadow prices.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →HOBA: Hierarchical On-Policy Bidding Agents for Adaptive Online Advertising
HOBA (Hierarchical On-policy Bidding Agents) is a novel hierarchical reinforcement learning framework that enhances online advertising bidding systems by improving adaptability and reducing hyperparameter tuning costs. It utilizes a for hyperparameter inference, a SARSA agent for expert model selection, and a dynamic expert pool for bid execution, achieving a +3.6% increase in target cost during large-scale deployment and outperforming state-of-the-art baselines on AuctionNet.