The Shadow Price of Reasoning: Economic Perspective on Optimal Budget Allocation for LLMs
Quick Answer
The paper introduces CLEAR, a method for optimal budget allocation in LLMs, improving global accuracy by up to 3x in resource-scarce scenarios.
Quick Take
By reallocating resources from insolvent to solvable queries, CLEAR enhances the Pareto frontier of token cost versus accuracy in reasoning tasks.
Key Points
- CLEAR reallocates resources from insolvent queries to those near solvable thresholds.
- The method significantly improves the Pareto frontier of token cost versus accuracy.
- In experiments, CLEAR achieved up to a 3x improvement in global accuracy.
- The approach is grounded in economic principles of constrained optimization.
- Utilizes a shifted-surge function to model per-query reasoning utility.
Paper Resources
Article Excerpt
From source RSS / original summaryarXiv:2606. 03092v1 Announce Type: new Abstract: has emerged as a critical avenue for enhancing ' performance, yet real-world deployment is constrained by strict computational budgets. In this work, we formulate inference budget allocation as a global constrained optimization problem governed by economic principles.
By modeling per-query reasoning utility with a shifted-surge function, we derive an optimal allocation policy based on a global shadow price that equilibrates marginal utility under resource scarcity. Based on this theory, we propose Constrained Latent-utility Equilibrium Allocation for Reasoning (CLEAR). It performs rational abandonment and reallocates resources from insolvent queries to solvable queries near their emergence thresholds.
Extensive experiments on several reasoning tasks with different traffic streams demonstrate that CLEAR significantly improves the Pareto frontier of total token cost versus mean accuracy. In resource-scarce regimes, CLEAR achieves up to a 3x improvement in global accuracy compared to uniform allocation.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →HOBA: Hierarchical On-Policy Bidding Agents for Adaptive Online Advertising
HOBA (Hierarchical On-policy Bidding Agents) is a novel hierarchical reinforcement learning framework that enhances online advertising bidding systems by improving adaptability and reducing hyperparameter tuning costs. It utilizes a for hyperparameter inference, a SARSA agent for expert model selection, and a dynamic expert pool for bid execution, achieving a +3.6% increase in target cost during large-scale deployment and outperforming state-of-the-art baselines on AuctionNet.