A Deep Learning Approach to Diagnostic Classification of Prostate Cancer Using Pathology-Radiology Fusion

J Magn Reson Imaging 2021 Deep Learning 8 Explanations View Original
Original Paper (PDF)

Unable to display PDF. Download it here or view on PMC.

Plain-English Explanations
Pages 1-2
The Problem With Prostate Biopsies

Prostate cancer is the most commonly diagnosed cancer in adult men, making accurate early diagnosis critically important for patient outcomes.

The current standard for definitive diagnosis requires a tissue biopsy, an invasive procedure that carries real risks. Between 2% and 3% of patients who undergo prostate biopsy develop sepsis, a life-threatening complication involving organ dysfunction.

Radiologists currently use a system called PI-RADS (Prostate Imaging Reporting and Data System) to evaluate MRI scans, assigning scores from 1 to 5. While useful, PI-RADS relies on subjective visual assessment, introducing variability among different reviewers.

Pathologists grade biopsy tissue using the Gleason Score (ranging from 6 to 10) or the newer Grade Group system (1 to 5), both designed to measure how aggressive the cancer appears. High-risk tumors grow faster and require more urgent treatment than low-risk ones.

TL;DR: Prostate cancer diagnosis currently requires invasive biopsies with serious complication risks, creating a need for safer non-invasive alternatives.
Pages 1-2
Introducing AI-Biopsy: The Study's Goal

Researchers at Weill Cornell Medicine developed an AI model called AI-biopsy that analyzes MRI scans to classify prostate cancer without requiring tissue removal.

The system was designed to accomplish two tasks: first, distinguish cancerous tumors from benign tissue, and second, identify whether a detected cancer is high-risk or low-risk.

The core hypothesis was that cancer aggressiveness could be predicted directly from MR images by training a machine learning model on MRI data labeled with corresponding biopsy results, essentially teaching the algorithm to read what pathologists see in tissue samples.

A key practical goal was to provide a drag-and-drop web interface accessible to radiologists in real time, without requiring programming knowledge or manual image segmentation steps.

TL;DR: AI-biopsy is a deep learning model trained to classify prostate cancer risk from MRI scans, aiming to reduce the need for invasive biopsies.
Pages 3-4
Building and Training the AI Model

The study assembled a dataset of 400 patients from five sources: 228 imaged in-house at Weill Cornell Medicine and 172 from publicly available databases including the PROSTATEx Challenge and TCGA-PRAD collections.

For each patient, two radiologists selected a consistent sequence of seven axial MRI slices showing the prostate gland, yielding 2,800 total images standardized to 512x512 pixels. All images were labeled with their corresponding Gleason Score or Grade Group from biopsy results.

The AI architecture used was Google's Inception-V1 (GoogLeNet) convolutional neural network, pre-trained on the large ImageNet image database and then fine-tuned specifically for prostate MRI classification through transfer learning.

Two separate models were trained: Model 1 classified cancer versus benign tissue, and Model 2 distinguished high-risk from low-risk cancer. To handle unequal numbers of cases in each category, the team used Gaussian noise augmentation to artificially expand the minority class.

Performance was validated using 5-fold cross-validation, a technique where the dataset is split five times to test the model on data it has never seen during training, ensuring results are not overfitted to the training set.

TL;DR: A deep convolutional neural network was trained on 400 patients' MRI data labeled with biopsy results, using two separate models for cancer detection and risk stratification.
Pages 4-5
How the Algorithm Reaches Its Decisions

The AI system processes a sequence of seven MRI slices per patient rather than a single image. A voting mechanism then aggregates predictions across slices to reach a final patient-level decision.

For cancer detection (Model 1), if even one out of seven slices registers a cancer probability of 50% or higher, the patient is flagged as having cancer. This conservative threshold minimizes false-negative predictions, which would be the more dangerous type of error in a clinical setting.

For risk stratification (Model 2), at least two out of seven slices must show high-risk probability above 50% before the patient is classified as high-risk, striking a balance between sensitivity and specificity.

To make the AI interpretable, the team applied Class Activation Maps (CAM), a technique that highlights which regions of the MRI image most influenced the model's decision. This allows clinicians to visually confirm whether the algorithm was focusing on the prostate gland or irrelevant areas.

TL;DR: AI-biopsy uses a multi-slice voting system and visual heat maps to reach and explain cancer risk predictions in a clinically interpretable way.
Pages 5-6
Performance Results and Comparison With PI-RADS

Model 1 (cancer versus benign) achieved an AUC of 0.89 (95% confidence interval: 0.86-0.92), with sensitivity, specificity, and accuracy all around 81-82%. An AUC near 1.0 indicates near-perfect discrimination, making 0.89 a strong result.

Model 2 (high-risk versus low-risk) achieved an AUC of 0.78 (95% CI: 0.74-0.82). Performance was higher when distinguishing the most extreme cases (AUC of 0.86 for Grade Group 5 versus Grade Group 1) but lower for intermediate-risk borderline cases (AUC of 0.71).

On a blind test set of 28 patients with known PI-RADS scores, AI-biopsy correctly classified 75% of patients, while the conventional PI-RADS system correctly classified only 53.6% of the same patients.

Agreement with pathology results (the gold standard) was measured using Cohen's kappa: AI-biopsy scored 0.467 (moderate agreement), while PI-RADS scored only 0.195 (slight agreement), confirming the AI model's substantial advantage over the current clinical standard.

TL;DR: AI-biopsy outperformed standard PI-RADS scoring in classifying prostate cancer risk, correctly identifying 75% of patients versus 53.6% for PI-RADS.
Pages 6-7
What the Activation Maps Revealed

Analysis of the Class Activation Maps (CAM) provided an important insight into why the AI sometimes made errors. When predictions were correct, the model consistently highlighted the prostate gland itself as the decisive region.

When predictions were incorrect, the CAM analysis showed that the algorithm had failed to identify the prostate gland at all, suggesting the errors were largely due to image-level issues rather than fundamental flaws in the cancer detection logic.

This finding validates the scientific integrity of the approach: the model learned clinically relevant tissue features within the prostate, not spurious background patterns, which is a critical requirement for any diagnostic AI system.

TL;DR: Visualization tools confirmed the AI focused on the prostate gland when making correct predictions, validating that it learned clinically meaningful features.
Pages 7-8
Advantages Over Prior Deep Learning Approaches

Earlier deep learning studies for prostate cancer typically used pre-segmented or cropped MRI images, requiring a radiologist to manually identify and isolate the prostate before feeding images to the algorithm. AI-biopsy instead works directly on unsegmented whole-image slices, removing this time-consuming step.

Most prior studies also used small, homogeneous datasets from a single institution or imaging protocol. AI-biopsy was validated on a heterogeneous, multi-site dataset spanning five separate cohorts with different imaging equipment, biopsy techniques, and pathologist reviewers.

Research on deep CNNs suggests that heterogeneity across datasets diminishes at deeper network layers. While input images may vary significantly between institutions, the model's final decision layer appears to extract consistent, biologically meaningful signals despite these differences.

Compared to competing models like FocalNet (AUC 0.81) and ADC-based methods (AUC 0.79), AI-biopsy achieved comparable or superior performance while processing a broader range of Gleason Scores and requiring no manual preprocessing steps.

TL;DR: AI-biopsy improves on prior methods by working on raw unsegmented MRI images and demonstrating robustness across diverse, multi-institutional datasets.
Pages 8-9
Clinical Implications and Limitations

If adopted clinically, AI-biopsy could allow radiologists to use an MRI-first approach, reserving biopsies only for cases where the AI identifies a high probability of cancer. This could dramatically reduce the number of unnecessary invasive procedures and associated complications like sepsis.

The system is designed for practical use through a publicly accessible web interface at ai-biopsy.eipm-research.org, requiring no specialized computational knowledge. Radiologists can upload MRI scans and receive real-time risk assessments with highlighted regions of concern.

The study acknowledges several limitations. Labels were assigned by pathologists from different institutions who may have applied slightly different scoring criteria. The model also lacked access to clinical variables such as patient age or PSA levels, which are routinely used in clinical risk assessment.

The current model only uses T2-weighted MRI sequences, though the framework supports other imaging types. Expanding to multiparametric MRI incorporating diffusion and perfusion sequences could further improve accuracy in future versions.

TL;DR: AI-biopsy has real-world clinical potential via a free web interface, though future work should incorporate additional clinical variables and imaging modalities.
Citation: Open Access, . Available at: PMC8360022.