Cost-Governed RAG: Unified Per-Tenant Cost Attribution Across Retrieval and Generation in Multi-Tenant LLM Systems
Quick Answer
Cost-Governed RAG introduces a unified cost attribution system for multi-tenant LLMs, achieving 99.96% accuracy in cost tracking across 100 simulated tenants.
Quick Take
Cost-Governed introduces a unified cost attribution system for multi-tenant LLMs, achieving 99.96% accuracy in cost tracking across 100 simulated tenants. By integrating TurboVec with a governance gateway, it reduces retrieval infrastructure costs by 3.1-9.0x compared to traditional vector databases, while maintaining low telemetry overhead.
Key Points
- Achieves 99.96% cost attribution accuracy across 100 simulated tenants.
- Reduces retrieval infrastructure costs by 3.1-9.0x compared to managed vector databases.
- Utilizes TurboVec's deterministic memory formula for precise cost calculations.
- Telemetry overhead remains below 0.04% of query latency.
- Introduces a three-layer cost model for effective governance.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Enterprise Retrieval-Augmented Generation (RAG) deployments face a critical governance gap: while LLM generation cost is metered per token, the retrieval layer - vector memory, similarity compute, and embedding API calls - remains an unattributed shared cost, enabling invisible cross-subsidization among tenants. We present Cost-Governed RAG, an architecture that integrates a codebook-oblivious vector index (TurboVec) with a multi-tenant LLM governance gateway, creating a unified observability stack where embedding, retrieval, and generation costs are jointly attributable per tenant. The architecture exploits TurboVec's deterministic, closed-form memory formula to enable near-exact per-tenant retrieval cost calculation - a property unavailable in graph-based indexes with non-linear memory overhead. Deployed on Snowpark Container Services within a cloud data platform's governance boundary, the system achieves 99.96% end-to-end cost attribution accuracy across 100 simulated tenants (10M vectors, log-normal size distribution) with telemetry overhead below 0.04% of query latency. The architecture reduces retrieval infrastructure cost by 3.1-9.0x compared to managed vector database services under the pricing assumptions detailed in Section IV. We formalize a three-layer cost model and demonstrate that codebook-oblivious quantization enables deterministic per-tenant cost attribution while also removing the shared-codebook leakage surface present in trained quantizers - the latter observation being exploratory and subject to the limitations described in Section VII.
| Subjects: | Artificial Intelligence (cs.AI); Databases (cs.DB); Information Retrieval (cs.IR) |
| Cite as: | arXiv:2607.12188 [cs.AI] |
| (or arXiv:2607.12188v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2607.12188 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Navnit Kumar Shukla [view email]
[v1]
Mon, 13 Jul 2026 22:16:58 UTC (13 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →Automatic Ordinary Differential Equations Discovery For Biological Systems Using Large Language Model Powered Agentic System
The MEDA system utilizes large language models and symbolic regression to autonomously discover ordinary differential equations for biological systems, achieving strong structural recovery and biologically plausible models. It outperforms existing methods by integrating domain knowledge and mechanistic constraints, demonstrating effective retrieval and extrapolation capabilities.