Whole slide images (WSIs) are digital scans of tissue specimens that typically measure 100,000 by 100,000 pixels, creating multi-gigabyte files with billions of pixels per image. The widespread adoption of digital pathology scanners has made these images increasingly available, but their enormous size creates fundamental challenges for standard deep learning techniques that cannot process such large inputs directly.
Deep learning has transformed computer vision, enabling algorithms that automatically learn to extract discriminative features from raw image data without human-defined feature engineering. Applying these advances to WSIs holds the potential to improve diagnostic accuracy, reduce pathologist workload, standardize clinical practice, and uncover new prognostic information from tissue morphology.
However, WSIs present two distinct problems beyond sheer scale. The what problem refers to the challenge of visual understanding in the presence of high morphological heterogeneity between cases, artifact contamination, and typically small labeled datasets. The where problem refers to the challenge of identifying which regions within a gigapixel image are diagnostically relevant without exhaustive pixel-level annotation.
This mini-review surveys the interdisciplinary methods being developed to address both problems, focusing on how image patches are selected and how predictions from those patches are aggregated to produce clinically useful slide-level or patient-level classifications.
The most common type of WSI uses hematoxylin and eosin (H&E) staining, which highlights tissue morphology through color contrast between cell nuclei and cytoplasm. H&E staining is the clinical standard for tissue diagnosis and is widely available, making it the most practical modality for deep learning applications. However, H&E slides lack molecular information about individual cells, unlike immunofluorescence staining, which can label specific proteins but requires expensive reagents and scanners.
Labeled data is more difficult to obtain in digital pathology than in many other deep learning domains. While large-scale public image datasets exist for natural images, annotating pathology slides requires expert pathologists who are scarce and whose time is expensive. The annotation granularity required for strongly supervised deep learning, where every tumor cell must be individually labeled, is often prohibitively time-consuming for real clinical datasets.
Artifacts introduced during slide preparation and scanning are a persistent challenge. Sources include ischemia times, tissue fixation, microtome cutting irregularities, staining reagent variability, uneven illumination, focus problems, tissue tears and folds, and fluorescence bleed-through in multiplex imaging. For an AI system to work reliably across real patient populations, it must learn to ignore these artifacts rather than treating them as diagnostically meaningful signals.
There is currently no universal open file format for WSIs, unlike radiology which uses the standardized DICOM format. Different scanner manufacturers use proprietary file formats with different compression methods and resolutions, complicating dataset curation and sharing. This technical fragmentation slows progress compared to other imaging fields.
Because WSIs are too large to process as single inputs, virtually all successful deep learning approaches extract image patches, typically square regions of around 256x256 pixels, and analyze them individually. Tiles are extracted from tissue regions after background suppression, usually by thresholding a grayscale version of the slide. The way patches are selected is one of the most important methodological decisions, as it directly determines what information the model learns from.
When patch-level annotations are available, meaning pathologists have labeled which specific regions contain cancer, strongly supervised training is possible. Patches can be extracted directly from labeled regions, ensuring all training patches have correct class labels. This approach enables hard negative mining, an iterative process where the model's false positive predictions are added to the training set to improve specificity. Models trained this way have repeatedly matched or surpassed pathologist accuracy in competition benchmarks such as CAMELYON16 and CAMELYON17 for breast cancer metastasis detection.
In most real-world scenarios, only slide-level labels are available: the diagnosis recorded in the pathology report indicates whether the whole slide is positive or negative, but does not specify which regions contain cancer. This low annotation granularity requires weakly supervised approaches. Methods include multiple instance learning, which assumes at least one patch in a positive slide is diagnostically informative; unsupervised representation learning; reinforcement learning for intelligent patch selection; and attention mechanisms that learn to weight patches by their diagnostic relevance.
Patient-level labels represent the weakest supervision, where the ground truth is known only at the patient level but not for individual slides, since multiple tissue sections can be cut from a single specimen. This scenario is addressed using the same aggregation strategies as slide-level learning, further up the label hierarchy.
Once a model classifies individual patches, its predictions must be aggregated to produce a slide-level or patient-level outcome. Simple aggregation uses max-pooling (the slide is positive if any patch is predicted positive) or average-pooling (the slide score is the mean patch score). More sophisticated approaches train secondary machine learning models such as random forests or recurrent neural networks on features derived from patch-level predictions, enabling richer integration of spatial and quantitative information across the slide.
A fundamental limitation of patch-based analysis is that each patch is processed independently, preventing the model from seeing spatial relationships between patches. For example, while individual cell morphology can be assessed within a single patch, higher-level architectural features such as tumor shape, growth pattern, or the spatial relationship between tumor and stroma require analyzing larger tissue regions. Spatial modeling approaches that explicitly capture correlations between neighboring patches have been proposed, though these typically still require patch-level annotations.
Attention mechanisms represent an emerging solution to both the where and what problems simultaneously. An attention model learns to selectively focus on the most diagnostically relevant regions of the slide, progressively improving its region selection as it trains. Reinforcement learning approaches have been applied where a model selects patches at multiple magnification levels based on a low-resolution overview of the entire slide, mimicking how a pathologist scans at low power before zooming in on suspicious areas.
End-to-end models that jointly optimize patch localization and visual understanding, rather than treating them as sequential stages, are a particularly promising direction. These architectures avoid the limitation that errors introduced during patch selection cannot be corrected during later training stages. Approaches using recurrent visual attention networks process large patches sequentially, attending to different subregions at each step to build up a representation of the entire slide structure.
Deep learning with patch-level annotations represents the most mature approach for WSI analysis and is closest to clinical deployment. When fine-grained labels are available, patch-based learning with hard negative mining can match or exceed pathologist accuracy for well-defined classification tasks such as breast cancer metastasis detection. The CAMELYON challenge series has played an important role in benchmarking and driving progress in this area.
However, patch-level supervision may ultimately limit the potential of AI systems, since model performance is constrained by the quality and completeness of human-provided annotations. Annotations may be incomplete, inconsistent between annotators, or may not capture all diagnostically relevant information. Weakly supervised approaches that learn directly from diagnostic reports, without requiring pathologist-drawn annotations, offer the potential to train on far larger and more representative datasets.
The review identifies a key open research direction: most methods that work with slide-level labels focus primarily on solving the where problem, identifying which patches are most relevant, without substantially simplifying the visual understanding challenge itself. One notable exception used neural image compression to reduce WSIs to a low-dimensional latent representation before classification, effectively simplifying both what and where problems together. Whether this approach extends to harder tasks like prognosis prediction warrants further investigation.
Moving beyond simple cancer detection toward higher-order clinical tasks such as patient survivability prediction, recurrence risk, and treatment response assessment will require more powerful representations and larger datasets than currently available. Weakly supervised and unsupervised learning methods are particularly well-suited to these tasks, since ground truth is only available at the patient level and not at the pixel or patch level.
The field is moving away from strong supervision requiring detailed annotations toward approaches that can learn from the natural structure of pathology data. Slide-level diagnostic labels extracted from existing pathology reports are readily available at scale in hospital information systems, making weakly supervised learning increasingly attractive as the default approach for building large training datasets.
For WSI analysis to reach its full clinical potential, several challenges must be addressed: developing robust methods that generalize across institutions with different staining protocols and scanners; creating standardized open data formats to facilitate large dataset curation; and building models that can reliably ignore artifacts while extracting diagnostically meaningful tissue features.
The shift toward attention-based and end-to-end architectures that simultaneously select regions and learn visual representations offers a path toward more powerful and interpretable models. Interpretability is particularly important in clinical settings, where pathologists need to understand which tissue regions drove a model's prediction before they can trust and act on its output.
Ultimately, the combination of digital pathology's growing adoption in clinical workflows with advances in weakly supervised, multi-scale, and attention-based deep learning will enable computational pathology tools to support pathologists in delivering more accurate, reproducible, and efficient diagnoses for cancer patients worldwide.