Simple Token-Efficient Vision-Language Model for Case-level Pathology Synoptic Report Generation
Quick Take
A novel token-efficient vision-language model for generating pathology reports from whole-slide images achieves high ROUGE-L/METEOR/BLEU-4 scores while using only half a NVIDIA H100 GPU, significantly reducing memory and runtime requirements. This model utilizes a minimal architecture with a frozen encoder and a lightweight aligner, demonstrating improved efficiency in multi-WSI settings.
Key Points
- Model reduces average sequence length by up to 64x using 512x512 patches.
- Achieves high performance with ROUGE-L/METEOR/BLEU-4 scores in evaluations.
- Training is feasible with only half a NVIDIA H100 GPU.
- Extensive ablations identify choices that enhance robustness in multi-WSI scenarios.
- Provides a reproducible baseline for efficient pathology report generation.
Article Content
From source RSS / original summaryarXiv:2605. 30716v1 Announce Type: new Abstract: Generating clinically useful pathology reports for pathology cases from whole-slide images (WSIs) is challenging due to gigapixel resolution, long visual-token sequences, and the complexity of case-level reasoning, where a single case may contain multiple WSIs with heterogeneous tissues and ambiguous findings. We present a simple token-efficient vision--language model for case-level synoptic report generation that remains practical under constrained GPU memory.
Our architecture follows a minimal three-component design: a frozen pathology patch encoder, a lightweight two-layer MLP vision-language aligner, and a large language model decoder, with an explicit WSI marker token to separate slides within a case. Training proceeds in two supervised stages: (1) aligner-only WSI captioning using heterogeneous WSI-text pairs, and (2) case-level supervised fine-tuning on case-report pairs for structured report generation.
To reduce sequence length, we represent each slide using $512 \times 512$ patches at $5\times$ magnification, which reduces the average sequence length by up to $64\times$ times compared to the commonly used $20\times$ patches. Combined with efficient training techniques, we enable practical training with only half a NVIDIA H100 GPU. Across both training stages, our approach achieves high ROUGE-L/METEOR/BLEU-4 scores while being substantially more efficient in memory and runtime.
In AI-based evaluations, our model is consistently preferred over strong baselines. Extensive ablations characterize performance-efficiency trade-offs and identify simple choices that improve robustness in multi-WSI settings. Overall, this work provides a strong, reproducible baseline for efficient pathology report generation, lowering the barrier to multi-WSI VLM research under limited compute.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →Evi-Steer: Learning to Steer Biomedical Vision-Language Models through Efficient and Generalizable Evidential Tuning
Evi-Steer introduces a novel evidential tuning framework for BiomedCLIP, enabling efficient fine-tuning with only 0.11% parameter updates. It significantly enhances performance in few-shot learning and domain shifts across 15 biomedical imaging datasets, demonstrating robustness for clinical applications.
