FINESSE-Bench: A Hierarchical Benchmark Suite for Financial Domain Knowledge and Technical Analysis in Large Language Models
Quick Answer
FINESSE-Bench introduces a comprehensive benchmark suite with 3,993 questions for evaluating financial competencies in large language models (LLMs), addressing gaps in existing benchmarks.
Quick Take
FINESSE-Bench introduces a comprehensive benchmark suite with 3,993 questions for evaluating financial competencies in large language models (LLMs), addressing gaps in existing benchmarks. It includes exam-oriented datasets inspired by professional certifications and aims to assess domain breadth and performance under varying difficulty levels.
Key Points
- FINESSE-Bench consists of eight specialized benchmarks for financial domain evaluation.
- It features datasets inspired by CFA, CMT, and CFTe professional certifications.
- The benchmark assesses performance degradation as difficulty increases.
- An automated scoring scheme is included for evaluating freeform answers.
- FINESSE-Bench complements existing financial benchmarks for more robust evaluations.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Large language models (LLMs) are increasingly being applied to financial analysis, reporting, investment decision support, risk management, compliance, and professional training. However, robust evaluation of their domain competence in finance remains incomplete. Widely used open benchmarks such as FinQA, ConvFinQA, and TAT-QA have played an important role in advancing financial question answering and numerical reasoning, but they focus primarily on question answering over financial reports and do not provide an explicit hierarchy of professional difficulty. Broader resources, including FinanceBench, PIXIU, FinBen, and FLaME, expand the coverage of financial tasks, yet the problem of evaluating the transition from foundational knowledge to expert-level financial reasoning remains open. In this work, we present FINESSE-Bench, a suite of eight specialized benchmarks comprising 3,993 questions for hierarchical evaluation of financial competencies in LLMs. FINESSE-Bench combines exam-oriented datasets inspired by professional certifications (CFA-like Levels 1-3, CMT-like Level 2, and CFTe-like Level 1), applied trading task collections, and a Russian-language olympiad benchmark. This design enables evaluation of domain breadth, performance degradation as difficulty increases, the ability to solve computational tasks, and model behavior in specialized financial domains. We also describe a unified evaluation protocol covering multiple-choice questions, numerical answers, and short open-ended responses, together with an automated scoring scheme for freeform answers based on the LLM-as-judge paradigm. FINESSE-Bench is intended both as a complement to existing open financial benchmarks and as a tool for more substantive evaluation of professionally relevant financial competencies in large language models.
| Comments: | 21 pages, 10 tables, 2 figures |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2605.15482 [cs.CL] |
| (or arXiv:2605.15482v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2605.15482 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Dmitry Stanishevskii [view email]
[v1]
Thu, 14 May 2026 23:53:51 UTC (146 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Quantifying Prior Dominance in Systems
The study introduces the Normalized Context Utilization (NCU) metric to evaluate Retrieval-Augmented Generation (RAG) systems, revealing that Small Language Models (SLMs) outperform larger models in factual extraction. The findings indicate that traditional scaling laws yield diminishing returns, with a commercial API frequently failing against adversarial evidence due to systemic confidence collapse.