Hormone receptor positive (HR+), HER2 negative (HER2-) breast cancer is the most common molecular subtype, accounting for 50-79% of all breast cancers. It is characterized by expression of the estrogen receptor (ER) or progesterone receptor (PR) and absence of HER2 overexpression. While endocrine therapy has improved outcomes for this group, long-term recurrence remains a significant clinical challenge.
Precision treatment for HR+/HER2- breast cancer increasingly depends on understanding the molecular landscape of individual tumors -- including specific gene mutations, activation of cancer-related pathways, and immune microenvironment characteristics. However, laboratory methods to detect these features are expensive, slow, and not universally available, creating barriers to widespread personalized treatment.
Deep learning applied to whole slide images (WSIs) of H&E-stained tissue -- the same slides used for routine pathology -- offers a potentially transformative alternative. If a model can learn to predict molecular features and prognosis directly from tumor morphology, it could enable rapid, low-cost stratification of patients at the time of diagnosis, guiding treatment decisions without requiring additional laboratory testing.
This retrospective study included 421 HR+/HER2- breast cancer patients who underwent surgery at Fudan University Shanghai Cancer Center between 2013 and 2014. All patients had H&E-stained histological slides scanned at 40x magnification to generate digital WSIs, which were then cut into approximately 3.38 million 256x256-pixel image tiles for analysis.
The cohort was notable for its multi-omics depth: 358 patients had whole-exome sequencing (WES) data (for somatic mutations), 417 had RNA sequencing data (for gene expression and pathway analysis), and 379 had copy number variation (CNV) data. A core subset of 323 patients had all five data modalities simultaneously, enabling comprehensive cross-modal prediction evaluation.
The study tested prediction of four categories of features: (1) clinicopathological features including tumor grade and Ki-67 proliferation index; (2) somatic mutations with frequency above 4% in the cohort; (3) gene set enrichment analysis (GSEA) pathway scores for six cancer-related biological pathways; and (4) immunotherapy biomarkers including tumor-infiltrating lymphocytes (TILs), PD-1, PD-L1, and CD8 expression.
The analysis pipeline used two convolutional neural networks in series. The first was a tissue type classifier (previously validated by the same group) that categorized each image tile into one of five tissue classes: tumor, stroma, immune infiltrates, normal duct, or necrosis. Across the dataset, tumor tiles accounted for 37.9% of all tiles, while stroma made up 53.4%.
Different tissue types were selected as inputs for different prediction tasks. For example, somatic mutation prediction used only tumor tiles, while immunotherapy biomarker prediction used stroma and immune infiltrate tiles, and prognosis prediction used tiles from all five tissue types. This selective approach ensured that each model received biologically relevant tissue context for its specific prediction target.
The second CNN, based on ResNet-18, was trained to predict each target from the selected tile type. Tile-level prediction scores were averaged to produce a patient-level score. The model saved after the epoch with the best validation AUC was used for final test set evaluation. Class activation maps were generated to visualize which tile regions most influenced each prediction.
Among the clinicopathological features tested, histological grade prediction was strongest: Grade III tumors were predicted with a test AUC of 0.90 (95% CI: 0.84-0.97), and Grade II with AUC 0.82. Grade I was harder to distinguish (AUC 0.68 in the test set). The model's confidence in Grade III may reflect the dramatic morphological changes -- severe nuclear atypia, frequent mitoses, absence of glandular structures -- that distinguish it from lower-grade tumors.
The Ki-67 proliferation marker was predicted with test AUCs of 0.81 for low expression and 0.80 for high expression, using a cutoff of 15% staining. This is a clinically valuable result because Ki-67 assessment is notoriously subjective and labor-intensive when done manually. An image-based prediction tool could standardize Ki-67 evaluation and reduce inter-pathologist variability.
In contrast, tumor size category (T) and lymph node involvement category (N) were not reliably predicted from primary tumor H&E images (test AUCs 0.51-0.57). This makes biological sense -- T and N categories reflect anatomical extent of spread, not tumor cell morphology, meaning pathological images of the primary tumor provide limited information about these features.
Among somatic mutations, TP53 and GATA3 mutations both achieved test AUCs of 0.68. TP53 is associated with worse prognosis and resistance to endocrine therapy, while GATA3 mutations are linked to favorable prognosis in luminal breast cancer. Predicting these mutations from H&E tiles suggests their associated morphological changes -- mitotic activity, necrosis patterns, and glandular differentiation -- carry detectable visual signatures.
Among biological pathways, the G2-M checkpoint pathway was best predicted with a test AUC of 0.79. G2-M checkpoint activation is associated with increased metastatic potential. The PI3K/AKT/mTOR signaling pathway achieved AUC 0.63 in the test set. This pathway is a major therapeutic target in HR+/HER2- breast cancer -- drugs like alpelisib (a PI3K inhibitor) are approved specifically for patients with PI3K-activated tumors.
For immunotherapy biomarkers, stromal TILs were predicted with AUC 0.76 in the test set, intratumoral TILs with AUC 0.78, PDCD1 (PD-1) mRNA with AUC 0.74, and CD8A mRNA with AUC 0.71. These immune markers are critical for identifying patients likely to respond to immune checkpoint inhibitors. Predicting them from routine H&E slides could enable preliminary immune profiling without costly immunohistochemistry or gene expression assays.
Two prognostic modeling strategies were compared using the DeepSurv neural network (a deep learning extension of the Cox proportional hazards survival model). The first used only clinical variables (tumor T and N stage), achieving a cross-validation concordance index (C-index) of 0.75 for overall survival (OS) and 0.71 for relapse-free survival (RFS).
The integrated model, which combined clinical features with deep image features extracted from all five tissue types, achieved C-index values of 0.76 for OS and 0.73 for RFS. Critically, the integrated model produced substantially higher hazard ratios between high-risk and low-risk patient groups -- 4.57 for OS (vs. 2.47 for the clinical-only model) -- demonstrating that deep image features capture meaningful prognostic information beyond what stage alone provides.
Kaplan-Meier survival curves clearly showed separation between high- and low-risk groups stratified by both models, with the integrated model achieving highly significant log-rank p-values (p less than 0.001 for RFS). These results suggest that the visual texture and architecture of tumor and stromal tissue encode prognostic information that is complementary to and partially independent of clinical staging variables.
The study's targets were carefully chosen for their clinical relevance to HR+/HER2- breast cancer management. TP53 mutation predicts resistance to tamoxifen and aromatase inhibitors -- the backbone of endocrine therapy. Identifying TP53-mutated patients from pathology alone could prompt earlier consideration of alternative therapies. GATA3 mutation, conversely, defines a luminal-like subgroup with favorable prognosis where de-escalation of treatment might be considered.
The ability to predict PI3K/AKT/mTOR pathway activation from H&E images has direct therapeutic implications: PI3K inhibitors (alpelisib) and AKT inhibitors (capivasertib) are approved or in trials specifically for patients with PI3K/AKT pathway activation in HR+/HER2- disease. Pathway prediction from pathology could serve as a rapid pre-screening tool to identify which patients should undergo confirmatory molecular testing.
Immunotherapy prediction is particularly forward-looking. While checkpoint inhibitors are not yet standard therapy in HR+/HER2- breast cancer, clinical trials are actively evaluating them. The ability to predict TIL density, PD-1, and CD8 expression from routine H&E -- without additional staining -- could streamline patient selection for immunotherapy trials and eventual clinical deployment.
This study established a deep-learning workflow capable of predicting a broad spectrum of clinically meaningful features in HR+/HER2- breast cancer from routine H&E-stained whole slide images: histological grade (AUC 0.90), Ki-67 status (AUC 0.80-0.81), TP53 and GATA3 mutations (AUC 0.68), G2-M pathway activation (AUC 0.79), TIL levels and key immune biomarkers (AUC 0.71-0.78), and prognosis (C-index up to 0.76).
Key limitations include the absence of external validation -- all analysis was conducted on a single-institution cohort. Not all predicted targets achieved strong accuracy (some mutation predictions and certain pathway scores remained modest), and the authors note that future studies should explore cell-level feature analysis to potentially improve molecular prediction accuracy.
The planned next step is validation using the publicly available TCGA (The Cancer Genome Atlas) dataset, which would allow assessment of model performance across diverse populations and imaging conditions. If validated externally, this workflow could offer a low-cost, rapid primary screening tool for therapeutic target identification and prognostic stratification in the most common form of breast cancer.