Inject or Navigate? Token-Efficient Retrieval for LLM Analysis of Transactional Legal Documents
Quick Answer
The study compares full-corpus injection against two structured retrieval methods, NAVEMBED and NAVINDEX, for analyzing transactional legal documents.
Quick Take
NAVINDEX achieved a 1.61x smaller token footprint and 25% lower costs while maintaining competitive performance, scoring tied on all 18 benchmark questions. Cached injection is only cheaper when the corpus is under ten times the retrieval payload.
Key Points
- NAVINDEX scored tied on all 18 questions with a 1.61x smaller token footprint.
- NAVEMBED and NAVINDEX reduced input tokens by 17.3x and 29.9x, respectively.
- Cached injection is cost-effective only when corpus size is below ten times retrieval payload.
- The study utilized a 20-question benchmark with verified ground-truth answers.
- NAVINDEX provided a 25% lower dollar cost compared to full-corpus injection.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Answering questions over a set of transactional legal documents is most simply done by injecting the whole corpus into the LLM's context window on every query. That baseline maximises retrieval recall, but its token footprint scales with the corpus rather than the question, and long-context degradation scales with it. We report what it took to replace full-corpus injection in a legal-document analysis system, comparing it against two structured retrieval modes over our proprietary structure-aware chunking: embedding retrieval (NAVEMBED) and LLM navigation over a compact structured index (NAVINDEX). On a 20-question benchmark with verified ground-truth answers, a position-bias-controlled, reference-anchored pairwise judge scored semantic retrieval with reranking tied with injection on 16 of 18 document-bound questions (injection preferred on 2) while attending to 17.3x fewer input tokens (a general-text-embedding (GTE) configuration reaches 29.9x at a lower tie rate); both modes were judged tied on the 2 out-of-scope controls. NAVINDEX was judged tied on all 18 at a 1.61x smaller total token footprint, a ~56x smaller answering context, and 25% lower dollar cost. We derive a closed-form caching-crossover rule: cached injection is cheaper in dollars only while the corpus stays below roughly ten times the retrieval payload. Scope and uncertainty are quantified in Section 8.
| Comments: | 17 pages, 2 figures, 8 tables |
| Subjects: | Computation and Language (cs.CL); Information Retrieval (cs.IR) |
| ACM classes: | H.3.1; H.3.3; I.2.1; I.2.7 |
| Cite as: | arXiv:2607.05764 [cs.CL] |
| (or arXiv:2607.05764v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.05764 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Mourad ElSheraey [view email]
[v1]
Tue, 7 Jul 2026 02:42:06 UTC (62 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.