Breast Invasive Ductal Carcinoma Classification on Whole Slide Images with Weakly-Supervised and Transfer Learning

Cancers 2021 Histopathology 6 Explanations View Original
Original Paper (PDF)

Unable to display PDF. Download it here or view on PMC.

Plain-English Explanations
Pages 1-2
Automating Breast Cancer Diagnosis from Biopsy Slides

Invasive ductal carcinoma (IDC) is the most common form of breast cancer, accounting for a large proportion of the 2.26 million breast cancer cases diagnosed worldwide in 2020. IDC is a heterogeneous group of tumors with a wide range of microscopic appearances, making consistent diagnosis challenging even for experienced pathologists.

Core needle biopsy has become the standard method for non-operative breast cancer diagnosis. It is cost effective, provides sufficient tissue for histopathological evaluation, and can definitively distinguish IDC from benign lesions such as fibroadenoma in over 90% of cases. Because of its widespread use, core needle biopsy is an ideal target for AI-assisted pathology tools.

This study trained deep learning models to classify IDC versus benign lesions in digitized biopsy and surgical whole slide images (WSIs). The approach combined transfer learning from ImageNet and weakly supervised learning, meaning most WSIs required only a slide-level label rather than detailed cell-by-cell annotation by pathologists.

Models were evaluated on four independent test sets totaling over 1,600 slides from multiple hospitals in Japan and The Cancer Genome Atlas (TCGA), achieving AUCs of 0.95 to 0.99, demonstrating strong potential as a clinical decision support tool for pathologists.

TL;DR: Deep learning models trained with transfer and weakly supervised learning achieved AUCs of 0.95 to 0.99 for classifying invasive ductal carcinoma in breast biopsy and surgical whole slide images.
Pages 2-3
Study Design and Training Approach

The study collected 2,183 H&E-stained breast WSIs from three Japanese hospitals, comprising 1,154 biopsy and 1,028 surgical specimens, spanning IDC and a range of benign lesions including fibroadenoma, mastopathy, ductal hyperplasia, and normal tissue. An additional 100 WSIs were obtained from TCGA. All slides were scanned at 20x magnification using the same Leica Aperio AT2 scanner to ensure consistency.

The training set consisted exclusively of core needle biopsy specimens. The four test sets included one biopsy set and three surgical sets from different hospitals and the TCGA database, testing the model's ability to generalize across institutions and specimen types. Each WSI diagnosis was verified by at least two pathologists, with a senior pathologist resolving discordant cases.

The core innovation was a two-stage weakly-supervised training approach. In the first stage, tiles were randomly sampled from positive and negative WSIs. In the second stage, hard mining was applied: the model performed inference on each full slide in sliding window fashion, then selected the tiles with the highest probability of being IDC for focused retraining. This iterative process allows the model to progressively focus on the most diagnostically relevant tissue regions without requiring individual tile-level annotations.

Only 96 of the 191 IDC-positive training WSIs had loose pathologist annotations indicating regions of interest. The remaining positive and all negative WSIs provided only slide-level labels, dramatically reducing the annotation burden required for training. The partial fine-tuning approach was used to train the model: only the batch normalization affine parameters and the final classification layer were updated, resulting in just 63,329 trainable parameters from the EfficientNetB1 architecture.

TL;DR: A two-stage weakly supervised training strategy combined with partial fine-tuning of EfficientNetB1 allowed effective model training using mostly slide-level labels, dramatically reducing pathologist annotation burden.
Pages 4-6
Classification Performance Across Test Sets

The best-performing model, trained at 10x magnification with 512x512 pixel tiles using EfficientNetB1 (TL x10 512 B1), achieved AUCs of 0.980, 0.958, 0.994, and 1.000 on the four test sets respectively. The consistently high performance across different hospitals and specimen types demonstrates meaningful generalizability beyond a single institution.

In contrast, existing models pretrained on adenocarcinoma from other organs, including stomach, colon, lung, and pancreas, achieved substantially lower AUCs of 0.66 to 0.89 when applied directly to breast IDC without fine-tuning. The stomach poorly-differentiated adenocarcinoma model achieved the highest cross-organ transfer AUC of 0.894, yet this remained far below the breast-specific fine-tuned models. This finding shows that even structural similarities between cancer types do not allow straightforward cross-organ model transfer.

Training at 10x magnification slightly outperformed 20x on the biopsy test set, and larger tiles of 512x512 pixels performed better than 224x224 pixels. Lower magnification captures more tissue context per image, which may help the model identify architectural patterns characteristic of IDC such as invasive growth into surrounding stroma, rather than relying solely on high-resolution cellular features.

Prediction heatmaps overlaid on WSIs showed that the model correctly highlighted IDC-positive regions corresponding to areas marked by surgical pathologists, including small isolated clusters of IDC cells. In true positive cases, the model detected both dense invasive areas and scattered individual malignant cells, demonstrating sensitivity to IDC across its heterogeneous growth patterns.

TL;DR: The breast-specific fine-tuned model achieved AUCs of 0.95 to 1.00 across all test sets, vastly outperforming organ-specific pretrained models that scored only 0.66 to 0.89 without breast-specific fine-tuning.
Pages 11-12
Error Analysis: False Positives and False Negatives

All 10 false positive cases in the biopsy test set were fibroadenomas, a common benign breast tumor. Analysis of these misclassified slides revealed that the problematic fibroadenomas displayed proliferative epithelial changes, fibrocystic changes, and enlarged spindle-shaped stromal cell nuclei with pleomorphism, features that are morphologically similar to the invading cells, ductular structures, and cancer stroma seen in IDC.

The training set contained only 91 fibroadenoma cases compared to 131 in the test set, and diagnostic reports did not include detailed fibroadenoma subtype information, meaning the model may not have seen a representative range of fibroadenoma morphological variants. Future work should focus on fibroadenoma subtype characterization to improve the training set and reduce this specific false positive pattern.

A clinically important biological distinction between IDC and benign proliferations such as fibroadenoma is the presence or absence of myoepithelial cells. Benign breast glands are surrounded by a layer of myoepithelial cells, whereas IDC lacks this layer as it invades through the basement membrane. Pathologists use immunohistochemistry markers for myoepithelial differentiation to confirm equivocal cases, and incorporating such markers into AI systems could reduce false positive rates in challenging morphological scenarios.

False negative cases included a scirrhous-type IDC, a subtype where cancer cells are embedded in dense fibrous stroma that may reduce the density of identifiable malignant cells per tile. These cases highlight that certain IDC architectural variants remain challenging for patch-based classification approaches and may benefit from whole-slide context integration or multi-scale analysis.

TL;DR: All false positives were fibroadenomas with IDC-like proliferative features, and false negatives included scirrhous-type IDC, pointing to morphological overlap and rare architectural variants as the main error sources.
Page 12
Clinical Applications and Future Directions

The high AUCs achieved across multiple hospital test sets suggest that this type of deep learning model, once properly validated in a clinical setting, could serve as a second reader during pathology screening workflows. Rather than replacing pathologists, the model could flag likely IDC-positive slides for priority review or mark regions of interest to focus the pathologist's attention.

Japanese clinical guidelines indicate that an IDC diagnosis is sufficient at the core needle biopsy stage, without requiring further subtyping. This makes core biopsy a particularly practical application for AI assistance, as the binary classification task (IDC vs. benign) aligns well with what the clinical decision point requires. For surgical specimens, however, pathologists typically need more detailed subtype classification, which would require additional model development.

The weakly supervised approach used in this study is practically significant because it reduces the annotation burden on pathologists, who would otherwise need to label thousands of individual tiles. Using slide-level labels for most cases, with a small proportion of loosely annotated positive cases, still achieved near-expert diagnostic performance, making this approach scalable to larger datasets and new clinical settings.

Future improvements should explore incorporating immunohistochemistry data to help distinguish IDC from morphologically similar benign proliferations, expanding the fibroadenoma subtype representation in training data, and extending the approach to IDC subtype classification for surgical specimens. Integration with laboratory information systems could enable seamless deployment as a triage tool in routine pathology workflows.

TL;DR: The model is suited for clinical deployment as a second reader for core needle biopsy triage, with future work needed to extend it to surgical specimen subtyping and incorporate immunohistochemistry markers.
Page 12
Conclusions

This study demonstrates that transfer learning combined with weakly supervised learning can produce highly accurate IDC classifiers for breast biopsy and surgical WSIs, achieving AUCs of 0.95 to 0.99 across four independent test sets from different hospitals. The partial fine-tuning approach enabled effective learning with minimal trainable parameters, making the training computationally efficient.

The clear performance gap between organ-specific cross-transfer models (AUCs 0.66 to 0.89) and breast-specific fine-tuned models (AUCs 0.95 to 0.99) confirms that domain-specific adaptation is essential for histopathological AI tools, even when the target cancer type shares structural similarities with other adenocarcinomas. General pre-trained models cannot be directly applied to new organ sites without fine-tuning.

The weakly supervised approach, which requires only slide-level labels for most training cases, represents a practically important advance for building AI pathology tools in real-world hospital settings where detailed pixel-level annotations are difficult and expensive to obtain at scale.

With further clinical validation and extension to additional benign subtypes and IDC subtype classification, these models have genuine potential to improve the efficiency and consistency of breast cancer diagnosis from histopathological specimens, ultimately benefiting both pathologists and patients.

TL;DR: Breast-specific fine-tuned models achieved AUCs up to 0.99 across multi-hospital test sets, confirming that domain-specific adaptation and weakly supervised learning together enable practical, high-performance AI pathology tools.
Citation: Open Access, 2021. Available at: PMC8582388.