Cost-Governed RAG: Unified Per-Tenant Cost Attribution Across Retrieval and Generation in Multi-Tenant LLM Systems
Quick Answer
Cost-Governed RAG introduces a unified cost attribution system for multi-tenant LLMs, achieving 99.96% accuracy in cost tracking across 100 simulated tenants.
Quick Take
By integrating TurboVec with a governance gateway, it reduces retrieval infrastructure costs by 3.1-9.0x compared to traditional vector databases, while maintaining low telemetry overhead.
Key Points
- Achieves 99.96% cost attribution accuracy across 100 simulated tenants.
- Reduces retrieval infrastructure costs by 3.1-9.0x compared to managed vector databases.
- Utilizes TurboVec's deterministic memory formula for precise cost calculations.
- Telemetry overhead remains below 0.04% of query latency.
- Introduces a three-layer cost model for effective governance.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Enterprise Retrieval-Augmented Generation (RAG) deployments face a critical governance gap: while LLM generation cost is metered per token, the retrieval layer - vector memory, similarity compute, and embedding API calls - remains an unattributed shared cost, enabling invisible cross-subsidization among tenants. We present Cost-Governed RAG, an architecture that integrates a codebook-oblivious vector index (TurboVec) with a multi-tenant LLM governance gateway, creating a unified observability stack where embedding, retrieval, and generation costs are jointly attributable per tenant. The architecture exploits TurboVec's deterministic, closed-form memory formula to enable near-exact per-tenant retrieval cost calculation - a property unavailable in graph-based indexes with non-linear memory overhead. Deployed on Snowpark Container Services within a cloud data platform's governance boundary, the system achieves 99.96% end-to-end cost attribution accuracy across 100 simulated tenants (10M vectors, log-normal size distribution) with telemetry overhead below 0.04% of query latency. The architecture reduces retrieval infrastructure cost by 3.1-9.0x compared to managed vector database services under the pricing assumptions detailed in Section IV. We formalize a three-layer cost model and demonstrate that codebook-oblivious quantization enables deterministic per-tenant cost attribution while also removing the shared-codebook leakage surface present in trained quantizers - the latter observation being exploratory and subject to the limitations described in Section VII.
| Subjects: | Artificial Intelligence (cs.AI); Databases (cs.DB); Information Retrieval (cs.IR) |
| Cite as: | arXiv:2607.12188 [cs.AI] |
| (or arXiv:2607.12188v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2607.12188 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Navnit Kumar Shukla [view email]
[v1]
Mon, 13 Jul 2026 22:16:58 UTC (13 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →HOBA: Hierarchical On-Policy Bidding Agents for Adaptive Online Advertising
HOBA (Hierarchical On-policy Bidding Agents) is a novel hierarchical reinforcement learning framework that enhances online advertising bidding systems by improving adaptability and reducing hyperparameter tuning costs. It utilizes a for hyperparameter inference, a SARSA agent for expert model selection, and a dynamic expert pool for bid execution, achieving a +3.6% increase in target cost during large-scale deployment and outperforming state-of-the-art baselines on AuctionNet.