Think Big, Search Small: Where Capacity Matters in Hierarchical Search Agents?
Quick Answer
This study reveals that hierarchical search agents benefit from role factorization, with a significant performance boost in exact match scores from 4.5 to 8.6 points across six model scales.
Quick Take
This study reveals that hierarchical search agents benefit from role factorization, with a significant performance boost in exact match scores from 4.5 to 8.6 points across six model scales. The delegation role is identified as the capacity bottleneck, with scaling it improving performance by ~11 points, while execution scaling yields only ~2.6 points. A 1.7B-parameter executor shows competitive accuracy with fewer tokens, suggesting a focus on enhancing delegation capacity.
Key Points
- Role factorization outperforms single-agent baseline, improving exact match by 4.1 points.
- Scaling the delegation role enhances performance by ~11 points, indicating a capacity bottleneck.
- A 1.7B-parameter executor achieves competitive accuracy with 37% fewer tokens.
- Focus on delegation capacity can optimize hierarchical search agent design.
- Study conducted across five multi-hop QA benchmarks.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Large language model based search agents increasingly adopt multi-agent architectures in which a main agent decomposes a complex question into sub-queries and dispatches them to parallel sub-agents. However, existing systems instantiate all roles from a single model of identical scale, leaving open how model capacity should be distributed across roles. We factorize hierarchical search into three roles: a delegation role responsible for task decomposition, an execution role responsible for retrieval and evidence extraction, and an answer generation role held fixed as a confound control. We then conduct controlled capacity sweeps along the delegation and execution axes on five multi-hop QA benchmarks. The experiments yield three findings. First, role factorization consistently outperforms a single-agent baseline, improving exact match from 4.5 to 8.6 points across six model scales. Second, capacity sensitivity is asymmetric: scaling the delegation backbone improves EM by ~11 points, whereas scaling the execution sub-agent moves EM by only ~2.6 points, identifying decomposition as the capability bottleneck. Third, a 1.7B-parameter executor trained via quality-filtered trajectory distillation matches a frontier sub-agent in accuracy while consuming 37% fewer sub-agent tokens, advancing the Pareto frontier. These results suggest a concrete recipe for building hierarchical search agents: concentrate capacity at delegation and downsize execution without sacrificing accuracy. Our code is available at this https URL.
| Comments: | 21pages |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2607.07548 [cs.CL] |
| (or arXiv:2607.07548v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.07548 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: YiBo Zhao [view email]
[v1]
Wed, 8 Jul 2026 15:46:48 UTC (3,071 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Quantifying Prior Dominance in Systems
The study introduces the Normalized Context Utilization (NCU) metric to evaluate Retrieval-Augmented Generation (RAG) systems, revealing that Small Language Models (SLMs) outperform larger models in factual extraction. The findings indicate that traditional scaling laws yield diminishing returns, with a commercial API frequently failing against adversarial evidence due to systemic confidence collapse.