On-Device Deep Research at 4B: Exposure Bounds Faithfulness, Retrieval Bounds Coverage
Quick Answer
This study investigates citation faithfulness in a 4B generator on a 24 GB laptop, revealing that exposure significantly impacts faithfulness, improving it from 0.45 to 0.58 for retrieved sources.
Quick Take
This study investigates citation faithfulness in a 4B generator on a 24 GB laptop, revealing that exposure significantly impacts faithfulness, improving it from 0.45 to 0.58 for retrieved sources. However, trustworthy coverage remains low at 0.22 due to recall limitations, suggesting that increasing source exposure is essential before addressing retrieval recall.
Key Points
- Faithfulness improves from 0.45 to 0.58 with increased source exposure.
- Trustworthy coverage remains low at 0.22 regardless of exposure levels.
- Exposure costs approximately 235 output tokens for improved faithfulness.
- The study uses a fixed 4B generator on a 24 GB laptop.
- Cited claim faithfulness and trustworthy coverage are treated as separate metrics.
Paper Resources
📖 Reader Mode
~2 min readAbstract:On-device research agents search a corpus, read sources, and write a cited brief on a personal laptop. Whether their citations are faithful, and at what cost, is unmeasured for a deployable small model. This study fixes one 4B generator on a 24 GB laptop and asks what makes its citations faithful. It separates two quantities usually reported as one number. Cited claim faithfulness asks whether the cited source supports the claim. Trustworthy coverage asks whether the agent also cites the right sources. The study crosses how much of each source the generator sees, 400 against 1500 characters, with the quality of the sources supplied, gold papers against retrieved papers. Two levers fall out, and they act on different outcomes. Exposure sets faithfulness. More of each source lifts faithfulness from 0.45 to 0.58 on retrieved sources and from 0.37 to 0.58 on gold sources, and the two settings converge, so faithfulness is bound by exposure, not by whether the source is correct. The exposure lift is robust to a second, independent judge; the exact convergence is tight under the primary judge and only approximate under the second. Retrieval sets coverage. Trustworthy coverage stays near 0.22 on retrieved sources at any exposure, because recall is held near 0.40, so exposure cannot fix which sources are cited. The extra exposure costs about 235 output tokens. The practical recipe is to raise per source exposure first, cheaply, and then treat retrieval recall as the only remaining lever.
| Comments: | 13 pages, 2 figures, appendix |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Information Retrieval (cs.IR); Machine Learning (cs.LG) |
| Cite as: | arXiv:2607.12257 [cs.AI] |
| (or arXiv:2607.12257v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2607.12257 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Vinay Kumar Chaganti [view email]
[v1]
Tue, 14 Jul 2026 01:57:39 UTC (63 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →The Emerging Paradigm of Geospatial Foundation Models: From Pre-Training to Agentic Reasoning
The paper introduces Geospatial Foundation Models (GeoFMs), AI/ML models pre-trained on extensive geospatial datasets, enabling domain experts to fine-tune them for specific tasks. This paradigm shift democratizes access to advanced AI/ML while ensuring security, and proposes a framework for cost-effective adaptation strategies. The vision of Agentic Geospatial Reasoning is also presented, where Large Language Models orchestrate GeoFMs to automate complex analytical workflows.