SAGE: Semantic Explainability of Attention-Based Survival Models in Computational Pathology
Quick Answer
SAGE introduces a post-hoc framework for extracting global, language-grounded explanations from attention-based multiple instance learning (ABMIL) models in computational pathology.
Quick Take
It demonstrates the ability to recover established prognostic features and reveal cancer-specific biology across seven TCGA cancer cohorts, enhancing interpretability for pathologists and aiding in biomarker identification.
Key Points
- SAGE extracts global explanations from frozen ABMIL models using a pathology .
- It scores image patches against 25 histological concepts, revealing relationships to prediction risk.
- Applied to seven TCGA cancer cohorts, SAGE identified established prognostic features.
- Ablation studies confirm associations depend on learned attention, not just concept prevalence.
- SAGE offers a scalable, model-agnostic framework for understanding ABMIL survival models.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Attention-based multiple instance learning (ABMIL) is the predominant approach for slide-level prediction in computational pathology, yet its attention maps provide only local explanations: they indicate where a model focuses but not which histological features drive its predictions or how the model behaves across a patient cohort. We present Semantic Attention Global Explanations (SAGE), a post-hoc framework that extracts global, language-grounded explanations from a frozen ABMIL model. Using a pathology vision-language model, SAGE scores image patches against a dictionary of 25 histological concepts, aggregates these scores according to the model's learned attention, and quantifies how each concept relates to prediction risk across a cohort. Applied to survival prediction using seven TCGA cancer cohorts and three foundation models, SAGE recovered established prognostic features, such as the adverse association of necrosis, while revealing cancer-specific biology, including a favorable angiogenic signature in renal cell carcinoma consistent with known molecular subtypes. Ablation studies demonstrated that these associations depend on the model's learned attention rather than concept prevalence alone, and that the concept dictionary captures much of the prognostic information encoded by the foundation model features. Through semantically-grounded explanations, SAGE provides a scalable, model-agnostic framework for understanding what ABMIL survival models learn, enabling pathologists to interpret model behavior at the cohort level and offering the potential for biomarker identification.
| Comments: | Proceedings of the MICCAI Workshop on Interpretability of Machine Intelligence in Medical Image Computing (iMIMIC) |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2608.02803 [cs.CV] |
| (or arXiv:2608.02803v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2608.02803 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: William Lotter [view email]
[v1]
Mon, 3 Aug 2026 18:58:39 UTC (5,911 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
ProMoE-FL introduces a Prototype-conditioned Mixture-of-Experts framework for multimodal federated learning, effectively addressing missing modalities. It outperforms existing methods on four chest X-ray datasets, demonstrating superior feature synthesis capabilities in both homogeneous and heterogeneous settings.