PathReportEval: A Systematic Benchmark for Pathology Report Generation
Quick Answer
This paper shows that The PathReportEval framework standardizes pathology report generation evaluation, introducing the Clinical Report Quality Score (CRQS) to assess factual correctness.
Quick Take
It benchmarks four methods across three datasets (TCGA, HistAI, REG 2025) using encoders like CONCHv1.5 and UNI2-h, revealing significant discrepancies in clinical accuracy compared to traditional metrics.
Key Points
- Introduces a standardized benchmark for pathology report generation evaluation.
- Evaluates four methods using datasets TCGA, HistAI, and REG 2025.
- CRQS measures clinical fact coverage and hallucination rates effectively.
- Traditional metrics like BLEU and ROUGE often misrepresent report quality.
- Framework allows integration of new methods and datasets for future research.
Paper Resources
Source Excerpt
Pathology report generation from whole-slide images (WSIs) is a rapidly growing multimodal learning problem, yet progress is difficult to measure because existing studies use heterogeneous datasets, model settings, visual encoders, and evaluation protocols. Moreover, commonly used natural language generation metrics, including BLEU, ROUGE, and METEOR, primarily reward lexical similarity and often fail to detect clinically consequential errors such as omitted diagnoses, hallucinated findings, or
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →RF-Agent: A Practical Framework for Building Language Agents for RFIC Design
RF-Agent introduces a novel framework for RF circuit design using , creating a unique RF-domain reasoning dataset with over 11,000 samples. The study reveals that domain-specific supervised fine-tuning and semantic retrieval strategies significantly enhance RF reasoning performance, particularly for smaller models.