FinReportBench: Measuring and Improving Institution-Grade Financial Report Generation
Quick Answer
FinReportBench introduces a 35-item benchmark for evaluating institutional-grade financial report generation, highlighting gaps in report identity and completeness.
Quick Take
The study shows that while basic deliverability is nearly saturated across nine model families, significant improvements in report quality can be achieved through skill distillation, resulting in a 33.85-point increase in mean G1 scores.
Key Points
- FinReportBench evaluates financial report generation with a 35-item expert rubric.
- Gaps identified include report identity, institutional components, and visual delivery.
- Basic deliverability is nearly saturated, but report identity remains a bottleneck.
- Skill distillation improved mean G1 scores by 33.85 points across five model families.
- Code and benchmark artifacts are publicly available for further research.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Large language models can produce fluent financial analysis, but fluency alone does not establish whether a report is suitable for institutional delivery. We introduce FinReportBench, an expert-grounded benchmark for measuring and improving institution-grade financial report generation. Expert review reveals recurring gaps in report identity, institutional components, source discipline, and visual delivery. We derive a 35-item rubric through expert partial orders, multimodal evidence, and audits of decision boundaries, covering deliverability, report identity, and institutional completeness. Starting from 10,000 balanced Chinese and English financial-research source records, we curate 244 bilingual tasks across three research objects and two input tiers. Each task separates the public query, reconstructed research trajectory, and hidden source packet. Three independent judge families reproduce the expert partial order at near-ceiling rates, showing that bounded, observable criteria support reliable evaluation. Across nine model families, basic deliverability is nearly saturated, while report identity and institutional completeness remain the primary bottlenecks. The largest cross-model gaps concern generation-trace control, information density, and data discipline rather than basic report framing. We then use benchmark-guided skill distillation to turn recurrent failures into reusable generation and self-review constraints. Across five model families, the evolved skill improves mean G1 by 33.85 points and mean G2 by 13.83 points over paired no-skill runs while preserving G0 for every pair. Code and benchmark artifacts are available at this https URL.
| Comments: | 9 pages, 9 figures |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2608.04374 [cs.CL] |
| (or arXiv:2608.04374v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2608.04374 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yinghao Tang [view email]
[v1]
Wed, 5 Aug 2026 02:31:48 UTC (2,788 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.