Breast cancer remains one of the most prevalent and lethal malignancies in women globally. Accurate early detection of both molecular biomarkers (such as estrogen receptor, progesterone receptor, HER2, and Ki-67) and tumor-infiltrating lymphocytes (TILs) is critical for diagnosis, classification, grading, and prognosis. Whole slide images (WSIs), the digital scans of entire tissue sections at ultra-high resolution, have become central to modern digital pathology workflows. These images are typically stained with Hematoxylin and Eosin (H&E) for structural visualization or with immunohistochemistry (IHC) to highlight specific molecular targets. A single WSI can exceed one billion pixels, making manual analysis time-consuming, subjective, and difficult to scale across large patient populations.
The deep learning promise and its persistent obstacles: Deep learning has emerged as a powerful approach for automating WSI analysis, with detection-based techniques generating clinically useful outputs like cell counts, biomarker localization, and lymphocyte spatial distributions. However, applying deep learning to WSIs presents stubborn challenges. The sheer resolution demands substantial computational resources and creates design difficulties for standard model architectures. WSIs also exhibit a multi-scale nature, ranging from microscopic cell morphology to macroscopic tissue organization, which requires models that can learn across spatial hierarchies. Additional complications include sparse diagnostic features, inter-sample heterogeneity, staining variations across laboratories, and the scarcity of large, well-annotated datasets that limits supervised learning scalability.
Gaps in existing reviews: Many prior reviews in this domain remain narrowly focused, concentrating on segmentation techniques or algorithmic performance metrics without adequately addressing the clinical integration of detection methods. Critical issues such as dataset bias, computational burden, and real-world deployment feedback are frequently overlooked. This paper addresses that gap by conducting a PRISMA-guided systematic review of 39 peer-reviewed studies and 20 widely used WSI datasets published between 2020 and 2024, with a focus on detection-oriented methods and their practical clinical utility.
Three guiding research questions: The review is organized around three core questions: (1) What types of datasets are used for comprehensive breast cancer assessment using WSIs? (2) What are the main challenges associated with WSI-based breast cancer assessment? (3) How do WSIs impact the accuracy and reliability of advanced deep learning approaches? The authors also introduce a five-dimensional evaluation framework covering accuracy and performance, robustness and generalization, interpretability, computational efficiency, and annotation quality to provide a balanced, clinically aligned assessment of both established methods and recent innovations.
Search strategy: The authors searched 8 major bibliographic databases, including Scopus, IEEE Xplore, Web of Science, SpringerLink, ACM Digital Library, and ScienceDirect, using combinations of terms such as "breast cancer detection AND deep learning," "convolutional neural networks AND breast cancer," "lymphocytes detection OR biomarkers detection AND breast cancer," and "automated breast cancer diagnosis OR AI in breast cancer screening." The search was restricted to articles published between 2020 and 2024 to capture the most recent and relevant findings in this rapidly evolving field.
Screening and selection: The initial automated and manual search retrieved 417 academic publications. After eliminating 254 duplicate records, 163 unique publications were screened based on titles, abstracts, and keywords. At this stage, 115 papers were excluded due to lack of domain relevance or methodological inadequacy. The remaining 48 full-text articles were retrieved for in-depth evaluation, with 1 excluded for inaccessibility and 9 more removed for specific reasons: 3 did not focus on breast cancer or its primary clinical tasks, 3 lacked concrete details about deep learning algorithms, 2 relied solely on conventional pathology, and 1 failed to meet the minimum quality threshold.
Quality assessment: To ensure reliability, the review adopted the Standard Quality Checklist (SCQ) comprising 10 evaluation items. Only studies that met at least 7 of the 10 SCQ criteria were included. Among the final 39 studies, the quality distribution was strong: 10 studies received a perfect score (10/10), 15 scored 9/10, 11 scored 8/10, and 3 met the minimum threshold at 7/10. This rigorous filtering process ensures that only methodologically sound and dependable studies contributed to the final synthesis.
Study characteristics: The 39 included studies span the full 2020-2024 period, with research output accelerating sharply in recent years. The highest number of articles appeared in 2023 (13 publications) and 2024 (9 publications), followed by 2021 (10 papers), 2020 (4 papers), and 2022 (only 2 papers, which the authors attribute in part to the impact of the COVID-19 pandemic on research output). The studies were categorized by task type: lymphocyte detection (LD), biomarker detection (BD), or both, and by model architecture, covering CNNs, U-Nets, Transformers, GANs, and hybrid approaches.
Scale and diversity: The review catalogs 20 WSI datasets that collectively define the data landscape for breast cancer deep learning. At the foundation sits TCGA-BRCA, a large-scale public repository containing 3,111 H&E-stained WSIs from 1,086 female and 12 male patients, paired with matched gene expression data and clinical information. TCGA-BRCA has spawned numerous derivative datasets, including MoNuSeg, BCSS, LYSTO, and TIGER, each tailored for specific detection tasks. Specialized high-resolution datasets like PanNuke (200,000 nuclei across 19 tissue types) and BreakHis (9,109 images from 82 patients at magnifications ranging from 40x to 400x) provide focused resources for particular research goals.
Task-specific datasets: For lymphocyte detection, the LYSTO dataset offers 20,000 images from 43 patients with breast, colon, and prostate cancers, enabling cross-cancer lymphocyte assessment. PanopTILs (2023) provides annotations for 814,886 nuclei from 151 patients, specifically enhancing the understanding of TILs in breast cancer. For biomarker detection, SHIDC-BC-Ki67 contains 2,357 tru-cut biopsy images of invasive ductal carcinoma annotated for Ki-67 markers, while the HER2 Challenge Contest dataset offers 100 gigapixel WSIs annotated with expert pathologist HER2 scores. Mitosis detection is served by MITOS-ATYPIA 14 (2,400 high-power field images), AMIDA13 (606 images from 23 subjects), and MIDOG21 (280 WSIs scanned by 4 different devices to address scanner variability).
Evolution toward richer annotations and ethical practices: The datasets show a clear temporal progression toward more comprehensive, finely annotated resources. Earlier datasets like BACH (2018) provided 400 H&E-stained patches with pixel-level annotations, while more recent entries like TIGER (2022) include WSIs of HER2-positive and triple-negative breast cancer with annotations for lymphocytes, plasma cells, invasive tumors, and stroma. The newest dataset, AI-TUMOR (2024), contains 2,500 WSIs with pixel-level annotations and explicitly focuses on reducing biases in AI models and ensuring better generalizability. This evolution reflects a growing emphasis on multimodal data integration, patient demographic diversity, and ethical data collection practices.
Multimodal integration opportunities: Datasets that combine histopathological images with clinical and genetic information, such as TCGA-BRCA, provide opportunities for more thorough analysis by capturing complementary biological features. The NuClick dataset (871 images from 440 WSIs) supports interactive annotation with careful patient-level separation across training, validation, and testing splits, preventing data leakage. Together, these 20 datasets enable applications ranging from simple cell identification to complex tumor categorization and prognostic modeling, serving as benchmarks for model development and validation across the field.
Challenge 1 - Size and complexity of WSIs: A single whole slide image can be several gigabytes in size, containing millions of pixels that must be processed. This demands substantial computational resources, including high-performance GPUs and large memory capacities. The logistics of storing and managing large WSI datasets compound the problem. Standard deep learning architectures were not designed for inputs of this magnitude, requiring specialized patch-based processing strategies, multi-resolution pipelines, or memory-efficient designs that add significant engineering complexity to any detection system.
Challenge 2 - Variability in image quality and resolution: Inconsistencies in image quality and resolution across different scanners and datasets represent one of the most widely cited obstacles, referenced by 10 of the 39 reviewed studies. Different scanning devices, staining protocols, and laboratory conditions introduce variations that degrade model generalization. A model trained on WSIs from one institution may perform poorly on slides scanned by different equipment at another hospital. This variability in training data quality directly reduces detection accuracy and limits the transferability of models to new clinical settings.
Challenge 3 - Annotation difficulties: High-quality annotations for WSIs are labor-intensive, time-consuming, and inherently prone to variability between annotators. Expert pathologists must manually label structures at the cellular level, a process that is both expensive and difficult to scale. Inconsistent annotations introduce biases that affect model robustness and generalizability. Six of the reviewed studies specifically highlighted annotation challenges as a major bottleneck, with the problem being especially acute for tasks requiring pixel-level precision, such as lymphocyte detection in dense tumor microenvironments.
Challenge 4 - Feature extraction difficulties: The complex and subtle patterns present in breast cancer tissues are inherently challenging to capture accurately. Traditional feature extraction methods often fall short, necessitating advanced deep learning techniques. This challenge was the most pervasive, cited by 26 of the 39 reviewed studies. Cell morphology can vary dramatically between patients and even within a single slide. Distinguishing cancerous from benign structures requires models that can simultaneously capture fine-grained cellular details and broader tissue-level context, a multi-scale requirement that pushes the boundaries of current architectures.
CNN and U-Net dominance: Over the 2020-2024 period, Convolutional Neural Networks (CNNs) and U-Net architectures remained the dominant baselines for WSI-based breast cancer detection. CNNs were predominantly applied to biomarker detection tasks, where classification or sparse detection was needed to identify ER/PR/HER2-positive cells. U-Net architectures were widely adopted for lymphocyte detection due to their pixel-level precision and strong segmentation capabilities, which are critical for detecting densely distributed immune cells with indistinct boundaries. For tasks combining biomarker and lymphocyte detection, U-Net-based frameworks were still preferred for their ability to support multi-task outputs such as simultaneous localization and segmentation.
The Transformer inflection point (2022+): From 2022 onward, architectural diversification accelerated significantly. Transformer-based models gained traction, especially in biomarker detection, by leveraging self-attention mechanisms to capture long-range contextual dependencies in high-resolution WSI data. Hybrid approaches emerged around 2023, combining the spatial locality of CNNs, the generative robustness of GANs, and the global modeling power of Transformers. Examples include CNN+Transformer and GAN+CNN+U-Net architectures that enable more adaptive and domain-generalizable detection systems. Lighter or exploratory models like Multilayer Perceptrons (MLP), Multiple Instance Learning (MIL), and YOLO appeared after 2021, though their use remained limited due to challenges in dense detection and precise localization on WSIs.
Evaluation metric diversity: The metrics used to assess these models varied with task type and output granularity. Classification and sparse detection tasks typically employed AUC, accuracy, and F1-score. Segmentation-oriented models were assessed using Dice coefficient, Intersection over Union (IoU), and boundary-aware metrics. For multi-output detection models, task-specific metrics were reported independently, reflecting the complexity of comprehensive breast cancer assessment. Top-performing models in the reviewed studies achieved F1 scores ranging from 0.73 (AlexNet on Ki-67 detection) to 0.967 (Deep-CNN+FSRM on BreakHis), with accuracy values reaching 98.8% for lymphocyte detection tasks.
A Sankey diagram perspective: The review presents a Sankey diagram (Figure 4) that visualizes the dynamic interplay between publication year, detection task, and model architecture from 2020 to 2024. Two major trends emerge: first, a shift in detection focus from early emphasis on biomarkers to increasing attention on lymphocyte detection and then to frameworks addressing both targets simultaneously; and second, a transition in model design from dominant use of CNN and U-Net toward more sophisticated hybrid approaches, reflecting growing demands for richer spatial modeling and generalization across WSI domains.
Enhancing model performance: The most active area of optimization involves ensemble learning frameworks that integrate diverse architectures such as U-Net, GANs, and CNNs to improve detection accuracy and generalization. Multi-task learning paradigms allow models to jointly learn segmentation, classification, and grading within a unified architecture, leveraging shared representations to reduce overfitting. Multimodal data fusion strategies that combine genomic data with WSIs or integrate multi-level data from cellular and tissue levels have shown significant gains in diagnostic precision. Advanced convolutional modules, including residual blocks, parallel blocks, dilated blocks, and color deconvolution, enhance feature representation and multi-scale information capture. Multiple attention mechanisms (spatial, channel, and self-attention) improve the model's focus on clinically relevant regions.
Improving robustness and generalizability: Two primary strategies dominate robustness enhancement. Cross-disease data integration exposes models to pathological variations across multiple cancer types, enabling better discrimination in complex clinical environments. Semi-supervised learning effectively addresses the scarcity of annotated data by leveraging limited labeled datasets alongside large volumes of unlabeled data. However, cross-disease integration can introduce label noise and annotation inconsistencies, potentially causing domain shift and reduced detection specificity. Pan-cancer training models have shown performance declines in certain tasks, suggesting that disease-specific signals can be diluted in multi-cancer settings.
Increasing interpretability: Two main approaches have been applied to make WSI detection models more transparent. The Human-Interpretable Features (HIF) paradigm aligns model predictions with visually and diagnostically meaningful image features, bridging the semantic gap between model outputs and clinical understanding. Saliency-based visualization methods produce heatmaps that localize regions contributing most to the model's decisions. However, HIF strategies typically rely on predefined, handcrafted feature sets that may inadequately capture the complex representations encoded by deep networks. Saliency visualizations are susceptible to input perturbations and architectural variations, sometimes producing unstable and misleading attributions.
Computational efficiency and annotation quality: Efficiency improvements center on precise localization of regions of interest (ROI) through Gaussian kernel annotation and micro-block selection techniques, enabling models to focus on key features while reducing computational costs. Pre-training strategies accelerate convergence, improve initialization, and reduce parameter counts. For annotation quality, weak supervision learning enables feature extraction from limited or imprecise annotations, while segmentation map generation techniques create synthetic annotations to compensate for incomplete labels. Both approaches synergistically improve robustness, though weakly supervised models remain vulnerable to label noise and segmentation map generation can introduce bias in complex tumor microenvironments.
Resolution as a double-edged sword: The review reveals a fundamental dichotomy in WSI-based deep learning. High-resolution images provide the detailed information crucial for detecting subtle differences between normal and cancerous tissues, especially in early-stage cancers. However, this same resolution increases computational requirements dramatically. The five-dimensional evaluation framework introduced by the authors addresses this tension by requiring that any comprehensive assessment of a WSI detection algorithm balance accuracy and performance metrics, robustness and generalizability, interpretability and explainability, computational efficiency, and annotation quality simultaneously, rather than optimizing for any single dimension in isolation.
Quality, consistency, and standardization: Variations in staining, lighting, and scanner types introduce noise and artifacts that directly affect model predictions. The review finds that inconsistent quality across datasets can lead to systematic inaccuracies, making standardization techniques vital for any model intended for multi-institutional deployment. Models trained on diverse WSIs covering a range of tissue types and patient demographics are more likely to generalize well, but the lack of standardization across institutions continues to hinder performance. The authors stress that reliability across varied clinical circumstances depends on maintaining constant image quality.
The annotation bottleneck: High-quality annotations by expert pathologists are essential for training accurate models, but obtaining them remains resource-intensive. Poor annotations directly reduce model accuracy, making annotation quality improvement a key priority. The review documents a progression in the field toward addressing this through semi-supervised and weakly supervised approaches that reduce dependence on fully annotated datasets. However, the effectiveness of these approaches varies significantly depending on the complexity of the detection task, with lymphocyte detection in dense microenvironments posing the greatest challenge.
Clinical integration requirements: The framework emphasizes that technical criteria alone are insufficient. Algorithms must also meet clinical criteria, with a preference for methods that reduce reliance on resource-intensive annotations by performing well with minimal or semi-supervised learning. The synergistic evaluation of all five dimensions ensures that chosen algorithms are not only methodologically sound but also practical and useful in real clinical settings. Through this approach, researchers and clinicians can evaluate and select algorithms suited to the complex problems of WSI-based breast cancer detection, with the ultimate goal of improving diagnostic accuracy and patient outcomes.
What has been achieved: The review demonstrates substantial progress in deep learning-based breast cancer detection using WSIs between 2020 and 2024. Model accuracy has improved across all task types, with ensemble and hybrid architectures consistently outperforming single-model baselines. The field has moved from predominantly CNN-based approaches to sophisticated combinations of Transformers, GANs, and attention mechanisms. The 20 cataloged datasets have grown in scale, annotation quality, and multimodal richness. Multi-task learning has proven effective for jointly addressing segmentation, classification, and grading, while multimodal data fusion strategies combining genomic data with WSIs have enhanced both diagnostic precision and interpretability.
What remains unsolved: Despite these advances, significant challenges persist. Computational scalability remains a barrier for real-time clinical deployment, as many high-performing models require hardware resources unavailable in typical hospital settings. Interpretability is still largely addressed through post hoc methods (HIF, saliency maps) that lack standardized validation protocols and can produce unstable results. Annotation quality continues to bottleneck the field, with most datasets reflecting a single institution's staining protocols and demographic profile. The translation gap between research benchmarks and real-world clinical utility remains wide, with few studies validating their methods in actual clinical workflows.
Short-term priorities: The authors recommend focusing immediately on developing lightweight, interpretable architectures optimized for WSI-scale processing to support real-time, resource-aware deployment. Enhancing weakly supervised and semi-supervised learning frameworks through uncertainty modeling and confidence-guided label refinement represents a technically viable strategy for improving annotation robustness. These near-term improvements would lower the barrier to clinical adoption by addressing the two most immediate practical constraints: computational cost and annotation availability.
Medium and long-term vision: In the medium term, the priority should be designing domain-adaptive and resolution-consistent models that address data heterogeneity across institutions and staining variations. In the long term, the establishment of clinically validated interpretability protocols and the construction of large-scale, standardized WSI datasets should be pursued to support reproducibility, benchmarking, and translational impact. The authors emphasize that prioritizing these directions will facilitate a more effective alignment between algorithmic innovation and real-world clinical integration, advancing the role of AI in precision breast cancer diagnostics and ultimately improving patient outcomes.