Prostate cancer is the second most commonly diagnosed cancer in men worldwide, yet its clinical behavior is notoriously difficult to predict. The same cancer grade in two patients can behave very differently, reflecting the disease's underlying molecular and morphological heterogeneity.
Pathologists grade prostate cancer by examining tissue stained with hematoxylin and eosin (H&E), a standard dye that reveals cellular structures under the microscope. The Gleason grading system (and its newer equivalent, ISUP grading) scores how abnormal the tissue architecture appears, which correlates with cancer aggressiveness.
Underlying these visible tissue patterns are thousands of genes whose expression levels vary across different regions of the same tumor. Understanding how what tissue looks like relates to which genes are active in that tissue could unlock more precise ways to distinguish cancers that need treatment from those that do not.
A new technology called spatial transcriptomics makes this connection measurable by simultaneously capturing gene expression data and tissue location at fine spatial resolution, enabling researchers to map gene activity directly onto histology images.
Researchers from Uppsala University and collaborating institutions designed a workflow to automatically extract morphological features from H&E tissue slides using a pre-trained AI and then correlate those features with gene expression data from the same tissue locations.
The key innovation was using an AI model originally trained to classify cancer grades in prostate biopsies and repurposing it for a different task: extracting rich descriptive features about tissue regions without any additional training or manual annotation.
The study was conducted on seven tissue sections from a single patient who underwent radical prostatectomy for multi-focal prostate cancer, with spatial transcriptomics data obtained from approximately 23,000 measurement spots distributed across the slides.
The central hypothesis was that AI-extracted visual features would define tissue sub-regions that match both independent expert pathologist annotations and distinct genetic expression profiles, bridging two traditionally separate information streams.
The AI used was a published ensemble of 30 convolutional neural networks (Inception V3 architecture) previously trained by Strom et al. to classify prostate tissue in needle biopsies into four categories: benign, and Gleason grades 3, 4, and 5. This ensemble had been validated on over 7,000 biopsy cores.
Rather than using the network's final classification outputs, the researchers extracted penultimate layer activations, the rich intermediate feature representations computed just before the network makes its cancer grade prediction. These 2,048 features per network encode complex morphological information about the tissue patch being analyzed.
Each of the 23,000+ spatial transcriptomics spots on the slides was centered as a 598x598 pixel patch and fed through all 30 networks, producing a 2,048 x 30 feature matrix per spot. UMAP dimensionality reduction was then applied to compress this into a manageable 300-dimensional representation while preserving the most meaningful variation.
For clustering and visualization, the 300 features were further compressed to 10 dimensions, and Gaussian Mixture Model clustering was applied to define tissue sub-regions. A separate 3-dimensional compression produced color maps where similar colors indicate morphologically similar regions of tissue.
When the AI-derived color maps of tissue heterogeneity were visually compared to manual annotations by two independent expert pathologists, they showed strong concordance across all seven tissue sections. Distinct tissue types such as benign tissue, stroma, and different cancer grades appeared as distinct colors in the AI maps.
In several cases, the AI detected finer distinctions than the pathologists' annotations. In one section (section 5), a region that both experts had labeled uniformly as a single cancer grade (ISUP2) appeared as two differently colored sub-regions in the AI map, suggesting the network detected morphological differences not visible to the human eye.
In section 7, both pathologists had marked a region as equivocal, meaning they could not confidently classify it as benign or cancerous. The AI automatically generated a distinct cluster for this ambiguous region, potentially indicating it contains biologically meaningful tissue characteristics despite being visually borderline to experts.
The comparison was formalized using Dice similarity matrices measuring spatial overlap between AI clusters and manual annotations, providing quantitative evidence of the concordance. The overall agreement supported the biological validity of the unsupervised AI segmentation.
The most striking finding was that AI-defined morphological clusters corresponded to distinct gene expression profiles from the spatial transcriptomics data. Tissue regions that looked different to the AI also expressed different sets of genes, validating that the visual features captured biologically meaningful information.
In section 5, the two sub-regions within the ISUP2-annotated area that the AI separated morphologically also showed different gene expression factors. The lower sub-region correlated with factor 10, containing genes like H2AFJ (linked to tumor progression) and TRGC1 (a potential aggressiveness marker). The upper sub-region correlated with factor 11, containing AGR2, a gene associated with low-grade cancer.
This means the AI split a single pathologist-labeled region into two areas with different genetic makeups, suggesting that within an ISUP2 grade tumor there exists biological heterogeneity that morphology can detect but current grading systems do not capture.
At a more granular level, individual genes within each expression factor showed high Pearson correlations with specific morphological features, meaning the AI's visual representations could predict the spatial distribution of individual gene expression levels, comparable to results from models specifically trained to predict gene expression.
Several of the AI-detected sub-regions that differed from pathologist annotations turned out to have biologically significant gene expression signatures. A cluster around a small ISUP1 cancer region that appeared benign to pathologists showed high expression of TSPAN1, a gene involved in cancer cell migration and under androgen control.
Another region labeled as ISUP2 cancer harbored expression of PDLIM5, a gene known to facilitate tumor growth and cell migration by activating metabolic pathways. This suggests the region may be more aggressive than its Gleason grade implies.
In section 7, the ambiguous tissue region the AI created a separate cluster for showed elevated expression of DEPDC1, a gene linked to tumor growth and cell proliferation in prostate cancer. This retroactively validated the AI's detection of something biologically abnormal in a region experts could not classify.
These discoveries support the idea that morphological patterns visible in H&E slides encode more genetic information than traditional grading captures, and that AI can decode this hidden information to refine cancer classification beyond what current clinical practice achieves.
The most immediate clinical application would be using AI-defined morphological regions to help pathologists annotate tissue samples more efficiently. The AI first defines coherent regions, and then a pathologist reviews and labels them, reducing the time spent manually tracing boundaries across entire tissue sections.
More profoundly, if AI can reliably detect regions of differing genetic character within a single cancer grade, this could help distinguish indolent cancer that can be monitored from aggressive disease requiring immediate treatment, a critical challenge in prostate cancer management where overtreatment is a known problem.
A key practical advantage of this approach is that it requires no new training data. Any research group with access to a publicly available prostate pathology AI and spatial transcriptomics data could apply this same workflow, making it broadly accessible.
Limitations include the use of tissue from a single patient (due to the high cost of spatial transcriptomics) and the need for validation across larger cohorts and different institutions. Future directions include combining this approach with single-cell RNA sequencing to link morphological clusters to specific cell type populations.