Immunohistochemistry (IHC) is the technique of using antibodies to stain specific proteins in cancer tissue sections, making them visible under the microscope. The presence, absence, or quantity of these protein markers gives pathologists critical information about tumor biology, guides treatment selection, and predicts patient outcomes.
However, manually reading and scoring IHC-stained slides is time-consuming, and its results are susceptible to significant inter-observer variability -- the same slide scored by two different pathologists may receive different results. This inconsistency is a serious clinical problem because treatment decisions for patients with breast, prostate, colorectal, and other cancers often hinge directly on IHC scores.
Modern digital pathology has created an opportunity to address this problem. When slides are digitized into high-resolution whole-slide images (WSIs), deep learning algorithms can be trained to perform the same counting and classification tasks as pathologists -- potentially faster, more consistently, and without fatigue-related errors.
This study aimed to develop and validate deep learning models for three different types of IHC staining patterns -- nuclear, cytoplasmic, and membranous -- covering a range of clinically important proteins across four cancer types (colon, prostate, breast, and endometrial). Crucially, the models were validated not just for agreement with manual scores, but for their ability to predict actual patient survival outcomes.
The proteins targeted in this study are all clinically relevant biomarkers. Ki-67 is a marker of cell proliferation -- tumors with high Ki-67 are growing faster and generally have a worse prognosis. It is prognostically important in breast, colon, and prostate cancer, though automated scoring tools would help standardize its clinical use.
PMS2 and MSH6 are DNA mismatch repair proteins whose loss indicates microsatellite instability (MSI), a molecular subtype of cancer associated with better prognosis and responsiveness to immunotherapy. Automated detection of MSI through these markers is particularly valuable in endometrial and colorectal cancer.
In prostate cancer, PTEN is a tumor suppressor gene whose loss is one of the most common events in aggressive prostate cancer and strongly predicts recurrence. Cyclin B1 (CCNB1) and Mapre2 are regulators of cell division whose expression levels correlate with prostate cancer outcomes. Beta-catenin, CD44, and Flotillin1 are proteins involved in cell adhesion and the process by which cancer cells acquire invasive behavior.
For breast cancer, estrogen receptor (ER) and progesterone receptor (PR) are the foundational biomarkers guiding treatment -- their presence determines whether patients benefit from hormone-blocking therapies. Accurate automated measurement of these receptors is thus directly tied to treatment decisions affecting hundreds of thousands of women annually.
The researchers developed three separate deep learning models, each optimized for a different cellular location of IHC staining: the nuclear model for proteins expressed in cell nuclei (Ki-67, PMS2, MSH6, Cyclin B1, ER, PR), the cytoplasmic model for cytoplasm-located proteins (PTEN, Mapre2), and the membranous model for proteins at the cell membrane (beta-catenin, CD44, Flotillin1).
Training required meticulous manual annotation: for each model, expert pathologists manually outlined and labeled thousands of individual tumor cells in image tiles, creating ground truth datasets with up to 102,403 individually labeled objects. The nuclear model used tiles from colon cancer Ki-67 and PMS2 slides; the cytoplasmic model from prostate PTEN slides; and the membranous model from prostate beta-catenin slides.
Two different neural network architectures were deployed: Mask R-CNN (initially) and YOLOv5 for the final models. Both are instance segmentation and object detection frameworks capable of identifying individual cells, drawing their boundaries, and classifying them as staining-positive or negative. The use of cell-level rather than patch-level analysis makes the approach more biologically interpretable than methods that score regions without identifying individual cells.
Five large retrospective patient cohorts with long-term follow-up data were used: 262 colon cancer patients, 266 and 259 prostate cancer patients from two separate cohorts, 1,228 endometrial cancer patients, and 142 breast cancer patients. All slides were digitized at the highest available resolution. The overall framework was designed to test not just technical accuracy, but whether the AI scores could predict patient survival as effectively as manual expert scores.
In the internal test sets, the correlation between deep learning scores and manual cell counts was exceptionally high: the average R-squared correlation between the two observers was 0.93, while the average correlation between one observer and the deep learning model was 0.95 -- the AI was more correlated with the expert than the two experts were with each other.
When comparing categorized (dichotomized) scores in the validation datasets -- meaning classifying each patient's tumor as high or low for each protein -- the correct classification rate (CCR) averaged 91.5% for the nuclear model (range 76.9-98.5%), 85.6% for the cytoplasmic model (73.3-96.6%), and 78.4% for the membranous model (75.5-84.3%). All correlations were statistically significant at p less than 0.001.
A particularly notable finding was that in the development sets, the deep learning model showed higher agreement with observer 1 than the two human observers had with each other for all proteins analyzed. This directly addresses the inter-observer variability problem that motivates the development of automated tools.
The membranous model had the lowest CCR values, which is not surprising given that membranous staining is also the most challenging for human observers to assess -- the boundary between membrane and cytoplasm is often visually ambiguous. Despite this, the membranous model showed specific cases where it outperformed manual scoring in terms of clinical utility.
The most important validation came from survival analysis: do patients stratified by AI scores into high and low expression groups show different survival rates, mirroring what is seen when manual scores are used? For the nuclear model, AI Ki-67 scores significantly predicted cancer-specific survival in colon cancer (hazard ratio 2.03) and time to recurrence in prostate cancer (HR 1.81 for Ki-67, HR 2.10 for Cyclin B1), closely matching the hazard ratios obtained from manual scoring.
For microsatellite instability assessment in endometrial cancer -- a clinically critical determination -- the deep learning MSH6 and PMS2 scores provided prognostic stratification that was significant for both markers, with hazard ratios essentially identical to those from manual scoring (HR 2.65 for DL vs. 2.09 for manual for MSH6 loss; HR 1.43 for DL vs. 1.44 for manual for PMS2 loss). For MSI status overall, the DL score HR was 1.89 versus 1.66 for manual.
The cytoplasmic model (PTEN) showed particularly strong prognostic validation. In an entirely independent second prostate cancer cohort not used in training, the deep learning PTEN hazard ratio was 3.32 compared to 3.34 for manual scoring -- virtually identical. This demonstrated genuine generalizability to new patient populations.
A striking finding involved the membranous model for beta-catenin and CD44 in prostate cancer: DL scores provided statistically significant survival stratification (p less than 0.001 for beta-catenin, p = 0.007 for CD44) while manual scores in the same cohorts were either non-significant or only marginally significant. This suggests that in some cases, the deep learning model is more sensitive to clinically meaningful variation in membranous protein expression than human observers.
One of the most clinically significant findings was the cross-cancer and cross-protein generalizability of the models. The nuclear model was trained on colon cancer Ki-67 and PMS2, yet it performed well on prostate cancer Ki-67, prostate Cyclin B1, breast cancer ER and PR, and endometrial cancer MSH6 and PMS2 -- cancer types and proteins never seen during training.
This generalizability is not a trivial finding. It suggests that the model learned general features of nuclear staining patterns that are shared across tissue types, rather than memorizing tissue-specific patterns. This property -- essential for clinical deployment -- means a single trained model can potentially be applied across a hospital's entire pathology workflow without needing to retrain for each new cancer type or protein.
The authors note that the models performed somewhat less well on prostate cancer samples than on colon, breast, and endometrial samples, likely because the training data was predominantly from colon cancer. Including prostate cancer samples in the initial training set would be expected to improve prostate-specific performance -- an actionable direction for future development.
The study also highlights a limitation of relying solely on agreement with manual scores as a measure of AI quality. The beta-catenin example -- where the AI found significant survival associations that manual scoring missed -- suggests that the true criterion for evaluating AI pathology tools should be their correlation with patient outcomes, not just their correlation with human observers who themselves introduce noise and variability.
This study provides robust multi-cohort evidence that deep learning IHC scoring is a clinically viable alternative to manual pathologist scoring. The models achieved strong agreement with expert counts, generalized across cancer types and proteins, and preserved the prognostic significance of all validated biomarkers -- the key clinical criterion for any replacement of manual assessment.
From a healthcare system perspective, the implications are significant. Pathologist workforces are under increasing pressure globally, with diagnostic caseloads rising while the number of trained pathologists remains constrained. Automated IHC scoring could substantially reduce the workload for routine prognostic markers, allowing pathologists to focus their attention on more complex diagnostic questions.
Automated scoring also addresses a quality problem: the inter-observer variability documented in this and many prior studies means that the same tissue sample may lead to different treatment decisions depending on which pathologist reads it. Consistent, reproducible AI scoring eliminates this source of clinical uncertainty.
Future work should include prospective clinical validation, integration of automatic tumor area detection (currently only implemented for the cytoplasmic model), and expansion of training datasets to include more diverse cancer types. The path to clinical integration requires large multi-center validation studies, but the foundational evidence presented here strongly supports continued development of these tools as genuine clinical aids.