Prostate cancer is the second leading cause of cancer death in men. A key part of diagnosis and treatment planning relies on the Gleason grading system, first developed in 1966 and recognized by both the World Health Organization (WHO) and the International Society of Urological Pathology (ISUP). The system classifies prostate tissue into five histological patterns (Grades 1 through 5), and the final Gleason score is reported as the sum of the two most prominent patterns. For example, a score of 4 + 3 = 7 means the most dominant pattern is Grade 4 and the second most dominant is Grade 3.
The inter-observer problem: Manual Gleason grading is performed by pathologists examining tissue under a microscope, a process that is time-consuming, tedious, and subject to significant inter-observer variability. Studies have shown that even specialized pathologists disagree on grade assignments, particularly for intermediate-risk patterns like Gleason 3 and Gleason 4, which are clinically very difficult to distinguish. Highly trained specialists have better conformity rates, but such experts are not widely accessible, especially in resource-limited settings.
Prior automated approaches: Earlier machine learning methods for prostate grading relied on hand-crafted feature extraction followed by conventional classifiers. More recent deep learning methods, particularly convolutional neural networks (CNNs), can learn complex features directly from data without manual feature engineering. Previous deep learning work on Gleason grading includes studies by Kallen et al. on homogeneous tissue slides, Zhou et al. on distinguishing Gleason 3 + 4 from 4 + 3, and del Toro et al. on binary low-versus-high classification. Patch-based classifiers differentiating benign tissue from Grades 3, 4, and 5 have also been explored, along with UNet-based segmentation approaches by Bulten et al. and Ren et al.
This study's contribution: The authors propose a transfer learning approach using 15 pretrained CNN architectures to classify prostate cancer tissue microarray (TMA) images into three classes: benign, Gleason Grade 3, and Gleason Grade 4/5. Unlike previous work that used limited architectures or required image segmentation, this study systematically benchmarks a broad set of modern pretrained models on a consistently labeled dataset from 244 patients, annotated by six expert pathologists with majority voting.
The dataset consists of prostate cancer tissue microarray (TMA) images from 244 patients, prepared at the Vancouver Prostate Centre in Vancouver, Canada. The study was approved by the institutional Clinical Research Ethics Board (CREB No. H15-01064). Each TMA spot was annotated by six expert pathologists independently, who delineated cancerous regions and assigned Gleason grades (1 through 5) based on WHO/ISUP criteria. TMA spots without cancerous patterns were marked as benign.
Majority voting for ground truth: To create a unified label for each TMA image, the authors applied majority voting across the six pathologists' pixel-wise annotations. In cases where the even number of annotators produced a tie, the first pathologist's decision was used as the tiebreaker. This multi-expert labeling approach is a notable strength compared to studies that use a single pathologist's annotations, which can introduce bias from individual variability.
Grade distribution: The clinical dataset did not contain Grades 1 or 2, so Gleason scores ranged from 6 (3 + 3) to 10 (5 + 5). The authors grouped the data into three classes for classification: benign, Grade 3, and Grade 4/5 combined. Gleason pattern 3 describes well-formed and separated glands, pattern 4 includes fused and poorly formed glands, and pattern 5 involves poorly differentiated individual cells, cords, and linear arrays.
Clinical significance: By using six specialists and aggregating their annotations, the trained model effectively captures the combined expertise and experience of all six pathologists. This approach is designed to produce classifications that are more objective and reproducible than any single pathologist's assessment. The dataset from Karimi et al. and Nir et al. has been used in prior studies, allowing for direct performance comparisons.
The original TMA images had a resolution of 5120 x 5120 pixels, far too large to feed directly into pretrained CNN models (which typically accept inputs of 224 x 224 to 456 x 456 pixels). To address this, the authors divided each full image into 750 x 750 pixel patches using a sliding window with a step size of 375 pixels (half the patch size). This produced 169 patches per original image. Each patch was labeled according to the annotation in its central 250 x 250 pixel region, and patches containing no annotations or multiple conflicting annotations in the central region were discarded.
Patch distribution: The process yielded a total of 23,901 usable image patches distributed across the three classes: 11,806 benign patches, 4,703 Grade 3 patches, and 7,392 Grade 4/5 patches (with only 147 of those being Grade 5). This severe class imbalance, particularly the scarcity of Grade 5 samples, could bias the network toward over-representing the majority class during training.
Data augmentation: To counteract the class imbalance, the authors used augmentation to increase the number of samples in underrepresented classes. Augmentation techniques included rotation (up to 10 degrees) and height shifts (up to 10%). The class with the most members served as the reference count, and smaller classes were upsampled to match. This is a straightforward but effective strategy to ensure the network does not develop a weight bias toward benign patches, which comprised nearly half of the dataset.
Why patches, not whole images: The authors provide two justifications for the patch-based approach. First, different regions within a single TMA image may have different Gleason grades, so treating the entire image as a single label would be inaccurate. Second, compressing a 5120 x 5120 image down to, for example, 224 x 224 pixels for ResNet50 would destroy critical histological detail needed for grading.
Training a deep CNN from scratch requires enormous datasets, often millions of images. With only 244 patients and 23,901 patches, the authors turned to transfer learning, a technique where a CNN pretrained on a large general-purpose dataset (ImageNet, with 14 million images across 1,000 categories) is fine-tuned on the target medical imaging task. The pretrained weights serve as a strong initialization, allowing the network to learn domain-specific features with far less data than training from random initialization would require.
Architectures tested: The study evaluated 15 pretrained CNN architectures: EfficientNet B0 through B5 (six models), NASNetLarge, NASNetMobile, InceptionV3, ResNet-50, SeResNet50, Xception, DenseNet121, ResNeXt50, and Inception-ResNet-v2. Each architecture processes images differently. VGG16 uses simple cascaded convolutional layers. Xception uses channel-wise separable convolutions for more efficient feature extraction. InceptionResNetV2 combines inception modules with residual connections. DenseNet121 feeds each layer with features from all previous layers to combat overfitting. EfficientNet models use compound scaling to balance depth, width, and resolution for superior efficiency with fewer parameters.
Fine-tuning procedure: For each architecture, the authors removed all fully connected layers and replaced them with a global average pooling layer followed by a softmax classification layer for the three output classes. All 15 models were fine-tuned for 50 epochs using stochastic gradient descent (SGD) with a learning rate of 0.0001, Nesterov momentum of 0.9, and a batch size of 32. Categorical cross-entropy served as the loss function. Input images were resized to match each model's expected dimensions, ranging from 224 x 224 (ResNet50, DenseNet121, and others) up to 456 x 456 (EfficientNet B5).
Evaluation strategy: Five-fold cross-validation was used to provide a robust estimate of each model's capabilities. The key metrics were overall accuracy across the three classes (benign, Grade 3, Grade 4/5), per-class accuracy, and area under the ROC curve (AUC). This systematic comparison across 15 architectures on the same dataset and training protocol provides a fair head-to-head benchmark.
Among all 15 architectures, NASNetLarge delivered the best overall performance with an accuracy of 0.93 and an AUC of 0.98 for the three-class classification task (benign, Grade 3, Grade 4/5). The second-best model was Inception-ResNet-v2 with an accuracy of 0.91 and AUC of 0.96, followed closely by Xception at 0.90 accuracy and 0.95 AUC. ResNet-50 also performed well, achieving 0.89 accuracy and 0.90 AUC.
Per-class performance of NASNetLarge: Breaking down the results by class, NASNetLarge achieved 0.95 accuracy for benign tissue, 0.89 accuracy for Grade 3, and 0.92 accuracy for Grade 4/5. This is notable because Grade 3 classification is typically the most difficult category, where pathologists themselves show the highest disagreement. The model's ability to reach 0.89 on Grade 3 demonstrates that it captures the subtle histological distinction between well-formed glands (Grade 3) and fused or poorly formed glands (Grade 4).
EfficientNet family performance: The EfficientNet models (B0 through B5) performed considerably worse than the top architectures. EfficientNet B0 achieved 0.81 accuracy and 0.83 AUC, while B1 through B5 ranged from 0.66 to 0.79 accuracy. Surprisingly, larger EfficientNet models (B3 through B5) performed worse than B0, suggesting that the compound scaling approach did not translate well to this particular histopathology task with limited data. NASNetMobile, the lightweight variant of NASNet, reached only 0.80 accuracy and 0.86 AUC, far behind its larger counterpart.
Confusion matrices: The authors present confusion matrices for NASNetLarge across all five folds. In Fold 1, out of 2,361 benign patches, 2,241 were correctly classified (94.9%), with 66 misclassified as Grade 3 and 54 as Grade 4/5. For Grade 3, 842 out of 940 patches were correct (89.6%), with 22 misclassified as benign and 76 as Grade 4/5. For Grade 4/5, 1,367 out of 1,479 were correct (92.4%), with 24 called benign and 88 called Grade 3. Performance was consistent across all five folds, confirming the model's stability.
A direct comparison is possible because the authors used the same TMA dataset (from the Vancouver Prostate Centre) as two prior published studies. Karimi et al. used multiple CNNs combined through a logistic regression model for a binary classification task (Grade 3 vs. Grades 4/5), achieving 86% accuracy. Nir et al. applied a random forest classifier to the same low-versus-high grade distinction, reaching 79.4% accuracy.
This study's advantage: The current work by Gifani and Shalbaf achieved 93% accuracy on a more challenging three-class problem (benign, Grade 3, Grade 4/5), which includes the additional complexity of separating benign tissue from cancerous grades. This is a substantially harder classification task than the binary low-versus-high grade distinction used in the comparison studies, yet the NASNetLarge model still outperformed both prior approaches by a significant margin.
Why NASNetLarge excels: NASNetLarge was designed through neural architecture search (NAS), an automated process where a reinforcement learning controller discovers optimal network configurations. With an input size of 331 x 331 pixels (larger than the 224 x 224 used by ResNet50 and DenseNet121), NASNetLarge can capture finer histological details. Its architecture, which combines normal cells and reduction cells in a repeating pattern, appears well-suited to the multi-scale features present in prostate tissue, from individual gland morphology to broader architectural patterns.
Clinical context: The authors emphasize that automated Gleason grading could serve as a diagnostic tool in settings where highly trained pathologists are unavailable. By compacting the knowledge of six expert pathologists into a single model through the majority-voting training scheme, the system provides a level of consensus-based grading that would be impractical to achieve manually for every case.
Small, single-center dataset: The entire study is based on 244 patients from a single institution (Vancouver Prostate Centre). While five-fold cross-validation provides internal validation, there is no external validation on an independent dataset from a different institution, scanner, or patient population. This limits confidence in how well the model would generalize to clinical practice in different settings, where tissue preparation, staining protocols, and scanning equipment may vary.
Severe Grade 5 underrepresentation: Of the 23,901 patches, only 147 were Grade 5, which were merged with the 7,245 Grade 4 patches into a single Grade 4/5 class. This means the model was never truly evaluated on its ability to distinguish Grade 5, the most aggressive pattern, from Grade 4. A clinically deployed system would need to separate these grades, as Grade 5 carries a substantially worse prognosis and may warrant different treatment decisions.
Majority voting tiebreaker: The ground truth was established through majority voting among six pathologists, with ties broken by defaulting to the first pathologist's label. This tiebreaking rule introduces a subtle bias toward one annotator's preferences. With an even number of raters, a more robust approach might use a seventh expert or a consensus discussion to resolve disagreements.
Lack of whole-slide image evaluation: The study operates entirely at the TMA and patch level. Tissue microarrays are small, curated tissue cores that do not capture the full heterogeneity present in whole-slide biopsy images. In clinical practice, pathologists assess entire biopsy cores, which contain a mix of grades, benign tissue, and artifacts. The transition from TMA patches to whole-slide grading would require additional architectural considerations such as attention mechanisms or multiple-instance learning.
Multi-center validation: The most critical next step is validating the NASNetLarge model (and other top performers) on datasets from multiple institutions with different patient demographics, tissue preparation protocols, and digital scanning equipment. Multi-center studies would reveal whether the 0.93 accuracy and 0.98 AUC hold up outside the Vancouver Prostate Centre, or whether domain adaptation techniques are needed to maintain performance across sites.
Whole-slide image integration: Moving from TMA patches to whole-slide biopsy images is essential for clinical relevance. This would likely require architectures designed for gigapixel-scale images, such as multiple-instance learning frameworks or hierarchical attention networks. Campanella et al. have demonstrated that such approaches can achieve AUCs above 0.989 on datasets of over 24,000 slides, establishing a benchmark for the field.
Finer grade separation: Future work should separate Grade 4 from Grade 5 rather than combining them, as this distinction has direct treatment implications. Additionally, predicting the specific Gleason score (such as 3 + 4 vs. 4 + 3) rather than just the grade would provide more clinically actionable information. The difference between 3 + 4 = 7 and 4 + 3 = 7 places patients in different ISUP Grade Groups (2 vs. 3), which affects treatment recommendations.
Explainability and clinical integration: For pathologists to trust and adopt automated grading, the models need interpretability features such as attention maps or gradient-based visualizations that highlight which tissue regions drove the classification decision. Integration into digital pathology workflows as a second-opinion tool, rather than a replacement for pathologists, represents the most realistic near-term deployment scenario. The authors note that such a system could be particularly valuable in regions where specialist pathologists are scarce, democratizing access to expert-level grading.