SproutRAG: Attention-Guided Tree Search with Progressive Embeddings for Long-Document RAG
Quick Answer
SproutRAG introduces an attention-guided hierarchical framework for retrieval-augmented generation, enhancing information efficiency by 6.1% over existing methods.
Quick Take
It organizes sentence-level chunks into coherent units without relying on external , enabling multi-granularity retrieval through a binary chunking tree. The framework is end-to-end trained, demonstrating superior performance across diverse benchmarks in scientific, legal, and open-domain contexts.
Key Points
- SproutRAG organizes sentence chunks into progressively larger coherent units.
- The framework improves information efficiency by 6.1% on average over strong baselines.
- It uses learned attention to capture semantic document structure effectively.
- Hierarchical beam search retrieves candidates at multiple granularities.
- Code is available on GitHub for further research and implementation.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Retrieval-augmented generation (RAG) systems must balance retrieval granularity with contextual coherence, a challenge that existing methods address through LLM-guided chunking, single-level context expansion, or hierarchical summarization. These approaches variously depend on costly LLM calls during indexing or retrieval, limit context aggregation to a single granularity level, or introduce information loss through summarization. We present SproutRAG, an attention-guided hierarchical RAG framework that addresses this trade-off by organizing sentence-level chunks into progressively larger but semantically coherent units, using learned inter-sentence attention to construct a binary chunking tree. Unlike prior approaches that rely on external LLMs, fixed context expansion, or lossy summarization, SproutRAG learns which attention heads and layers best capture semantic document structure, enabling multi-granularity retrieval without additional LLM calls or compressed summaries. At retrieval time, SproutRAG uses hierarchical beam search to retrieve candidates at multiple granularities, capturing multi-sentence relevance beyond flat retrieval. The framework is trained end-to-end with a joint objective that improves both embeddings and tree structure. Experiments across four benchmarks spanning scientific, legal, and open-domain settings demonstrate that SproutRAG improves information efficiency (IE) by 6.1% on average over the strongest baseline. Code is available on this https URL.
| Subjects: | Computation and Language (cs.CL); Information Retrieval (cs.IR) |
| Cite as: | arXiv:2606.18381 [cs.CL] |
| (or arXiv:2606.18381v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2606.18381 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Amirhossein Abaskohi [view email]
[v1]
Tue, 16 Jun 2026 18:28:00 UTC (3,350 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.