JASPR: Joint Spatial Representation learning of histology and spatial genomics for improved virtual genomic screening and clinical prognostication
Quick Answer
JASPR is a self-supervised deep learning framework that integrates hematoxylin and eosin (HE) images with spatial transcriptomics (ST) data, enhancing predictions of 9,248 genes in breast cancer.
Quick Take
JASPR is a self-supervised deep learning framework that integrates hematoxylin and eosin (HE) images with spatial transcriptomics (ST) data, enhancing predictions of 9,248 genes in breast cancer. By learning joint representations and incorporating spatial context, JASPR significantly improves prognostic outcomes compared to traditional methods.
Key Points
- JASPR employs a cross-modal reconstruction objective for HE and ST data integration.
- The framework captures universal spatial properties while encoding modality-specific features.
- Validation on breast cancer datasets shows improved HE-based gene predictions.
- JASPR enhances prognostic value for breast cancer outcomes significantly.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Recent studies have shown that spatial properties of tumors are critical for understanding disease biology and predicting patient outcomes. These spatial properties are increasingly uncovered through complementary modalities: spatial transcriptomics (ST) captures spatially-resolved molecular states, while hematoxylin and eosin-stained whole slide images (HE) reveal tissue morphology. While approaches are emerging to fuse these modalities, effective methods that learn not only joint representations but also incorporate spatial context across modalities are lacking. Here, we present JASPR (Joint Spatial Representation learning), a self-supervised deep learning framework that integrates HE images and ST data through a cross-modal reconstruction objective that incorporates spatial context within HE images and ST profiles. It employs shared modules to capture universal spatial properties across modalities, while modality-specific experts encode features unique to morphological and genomic data. We train and validate JASPR on breast cancer datasets, demonstrating that its learned joint representation substantially improves HE-based prediction of 9,248 genes and provides prognostic value for breast cancer outcomes.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2606.28395 [cs.CV] |
| (or arXiv:2606.28395v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2606.28395 arXiv-issued DOI via DataCite |
Submission history
From: Marija Pizurica [view email]
[v1]
Wed, 24 Jun 2026 00:35:49 UTC (11,913 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CV
See more →LLM-Guided ANN Index Optimization for Human-Object Interaction Retrieval
A phase-aware LLM agent optimizes human-object interaction retrieval, outperforming Optuna TPE by 33.3% and VDTuner by 34.2% on the HICO-DET benchmark. This method enhances throughput by 15.3x over UniIR and demonstrates strong transferability across vector database management systems.