On-Device Deep Research at 4B: Exposure Bounds Faithfulness, Retrieval Bounds Coverage
Quick Answer
This study investigates citation faithfulness in a 4B generator on a 24 GB laptop, revealing that exposure significantly impacts faithfulness, improving it from 0.45 to 0.58 for retrieved sources.
Quick Take
However, trustworthy coverage remains low at 0.22 due to recall limitations, suggesting that increasing source exposure is essential before addressing retrieval recall.
Key Points
- Faithfulness improves from 0.45 to 0.58 with increased source exposure.
- Trustworthy coverage remains low at 0.22 regardless of exposure levels.
- Exposure costs approximately 235 output tokens for improved faithfulness.
- The study uses a fixed 4B generator on a 24 GB laptop.
- Cited claim faithfulness and trustworthy coverage are treated as separate metrics.
Paper Resources
📖 Reader Mode
~2 min readAbstract:On-device research agents search a corpus, read sources, and write a cited brief on a personal laptop. Whether their citations are faithful, and at what cost, is unmeasured for a deployable small model. This study fixes one 4B generator on a 24 GB laptop and asks what makes its citations faithful. It separates two quantities usually reported as one number. Cited claim faithfulness asks whether the cited source supports the claim. Trustworthy coverage asks whether the agent also cites the right sources. The study crosses how much of each source the generator sees, 400 against 1500 characters, with the quality of the sources supplied, gold papers against retrieved papers. Two levers fall out, and they act on different outcomes. Exposure sets faithfulness. More of each source lifts faithfulness from 0.45 to 0.58 on retrieved sources and from 0.37 to 0.58 on gold sources, and the two settings converge, so faithfulness is bound by exposure, not by whether the source is correct. The exposure lift is robust to a second, independent judge; the exact convergence is tight under the primary judge and only approximate under the second. Retrieval sets coverage. Trustworthy coverage stays near 0.22 on retrieved sources at any exposure, because recall is held near 0.40, so exposure cannot fix which sources are cited. The extra exposure costs about 235 output tokens. The practical recipe is to raise per source exposure first, cheaply, and then treat retrieval recall as the only remaining lever.
| Comments: | 13 pages, 2 figures, appendix |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Information Retrieval (cs.IR); Machine Learning (cs.LG) |
| Cite as: | arXiv:2607.12257 [cs.AI] |
| (or arXiv:2607.12257v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2607.12257 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Vinay Kumar Chaganti [view email]
[v1]
Tue, 14 Jul 2026 01:57:39 UTC (63 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →HOBA: Hierarchical On-Policy Bidding Agents for Adaptive Online Advertising
HOBA (Hierarchical On-policy Bidding Agents) is a novel hierarchical reinforcement learning framework that enhances online advertising bidding systems by improving adaptability and reducing hyperparameter tuning costs. It utilizes a for hyperparameter inference, a SARSA agent for expert model selection, and a dynamic expert pool for bid execution, achieving a +3.6% increase in target cost during large-scale deployment and outperforming state-of-the-art baselines on AuctionNet.