Deep Learning for Fully Automatic Tumor Segmentation on DCE MRI of Triple-Negative Breast Cancer

Cancers 2023 MRI Analysis 7 Explanations View Original
Original Paper (PDF)

Unable to display PDF. Download it here or view on PMC.

Plain-English Explanations
Pages 1-2
The Segmentation Bottleneck in TNBC Treatment Monitoring

Triple-negative breast cancer (TNBC) accounts for approximately 15% of all breast cancers but contributes to about 40% of breast cancer-related deaths -- a disproportionate toll that reflects its aggressive biology and resistance to the receptor-targeted therapies that have transformed outcomes in other breast cancer subtypes. The standard treatment approach is neoadjuvant systemic therapy (chemotherapy given before surgery to shrink the tumor), followed by surgical removal of the remaining disease.

A critical challenge is that only about 50% of TNBC patients achieve a pathological complete response -- meaning no viable cancer cells remain after neoadjuvant therapy -- a result that strongly predicts good long-term outcomes. The other half have varying degrees of residual tumor. Non-invasive imaging methods that can accurately measure tumor size and detect early signs of treatment response are therefore highly valuable for guiding therapy decisions and avoiding unnecessary side effects from ineffective regimens.

Dynamic contrast-enhanced (DCE) MRI is a powerful tool for TNBC monitoring. It uses a contrast agent injected into the bloodstream to highlight tumor tissue based on blood vessel permeability and density -- characteristics that differ between cancer cells and normal tissue. DCE-MRI captures these enhancement patterns over time, generating information about tumor size, vascularity, and response to treatment that plain anatomic imaging cannot provide.

Before any quantitative measurement can be extracted from DCE-MRI, the tumor must be precisely outlined in three dimensions -- a process called tumor segmentation. Manual segmentation by radiologists is slow, requires hours of expert effort per patient, and varies between readers or even when the same radiologist repeats the task. This variability limits reproducibility and makes high-throughput monitoring of many patients across treatment timepoints impractical. Automating this step with deep learning was the central goal of this study.

TL;DR: TNBC requires close monitoring with DCE-MRI during chemotherapy, but manual tumor segmentation is too slow and variable for large-scale treatment response analysis -- deep learning automation is urgently needed.
Pages 3-5
nnU-Net Framework and a Multi-Timepoint Dataset

The study drew on data from an ongoing prospective clinical trial (NCT02276443) at MD Anderson Cancer Center, enrolling 301 patients with biopsy-confirmed stage I-III TNBC. Each patient underwent DCE-MRI at three treatment timepoints: before treatment (baseline, BL), after two cycles of chemotherapy (C2), and after four cycles (C4). After excluding cases with inflammatory breast cancer, failed manual segmentation, or complete tumor disappearance, 744 usable imaging datasets were available -- 285 from BL, 207 from C2, and 252 from C4.

Imaging used a 3T MRI scanner with a bilateral breast coil. The DCE protocol acquired 3D T1-weighted images dynamically over time before and after contrast injection, generating a time-series of images that capture how the contrast agent flows into and out of tumor tissue. From this time-series, three types of derived images were calculated and evaluated as potential model inputs: PEI (positive enhancement integral, measuring total contrast uptake), SER (signal enhancement ratio, comparing early and late enhancement), and MSI (maximum slope of increase, measuring how quickly contrast enters). Additionally, a simple subtraction image was created by subtracting the pre-contrast image from the early post-contrast image -- highlighting tissue with significant contrast uptake.

The deep learning framework used was nnU-Net ("no-new-Net"), a self-configuring segmentation system based on the standard U-Net architecture. Unlike traditional deep learning systems that require extensive expert tuning of hyperparameters (learning rate, batch size, image preprocessing, network depth), nnU-Net automatically determines these settings by analyzing the characteristics of the training dataset -- image resolution, size distribution, intensity statistics -- through a rules-based system and dictionary lookup. This makes it highly adaptable to new datasets without requiring deep machine learning expertise to configure.

Ten models were systematically constructed to compare different input types and training strategies. Five models used BL data only with different input combinations (subtraction image alone, PEI alone, SER alone, MSI alone, all four combined). Three compared training on single timepoints (BL only, C2 only, C4 only). The final key model, nnU-Net_3tpt, was trained on data from all three timepoints combined. Two additional models examined whether including or excluding biopsy clips and central tumor necrosis from the reference segmentation masks affected performance.

TL;DR: Ten nnU-Net variants were systematically compared using 744 DCE-MRI datasets from 301 TNBC patients across three treatment timepoints, testing different image types and multi-timepoint training strategies.
Pages 6-9
Subtraction Images and Multi-Timepoint Training Outperform All Alternatives

For input image type, the simple subtraction image (pre-contrast subtracted from early post-contrast) achieved the best segmentation performance -- equal to or better than all three computed parametric maps (PEI, SER, MSI) and equivalent to the combination of all four. On the BL test set, the subtraction-based model achieved a Dice similarity coefficient (DSC) of 93% and sensitivity of 96%. This finding is practically important: subtraction images are straightforward to generate on any clinical workstation, while parametric maps require specialized software -- yet the simpler input performs better.

The nnU-Net_3tpt model, trained on data combined from all three treatment timepoints, outperformed models trained on any single timepoint for both DSC and sensitivity (statistically significant, p less than 0.05 for all comparisons). Training with three timepoints effectively tripled the dataset size while introducing more diverse tumor morphologies -- smaller, more irregular tumors at later timepoints after chemotherapy response -- making the network more robust. On the BL test set, nnU-Net_3tpt achieved DSC of 93% and sensitivity of 96%, matching or exceeding the best recently published breast cancer segmentation models trained on substantially larger datasets from other institutions.

Tumor size strongly influenced segmentation accuracy. The model performed significantly better on tumors larger than 2 cm than on tumors 2 cm or smaller, a pattern seen consistently across timepoints and also reported in other segmentation studies. The reduced performance on smaller tumors is expected: smaller tumors are harder to delineate, represent a proportionally larger contribution from boundary uncertainty, and have less contrast with surrounding tissue after partial treatment response. The mean tumor diameter decreased across timepoints (3.4 cm at BL, 2.6 cm at C2, 2.1 cm at C4), partly explaining the somewhat lower DSC at later timepoints.

For tumor size estimation, nnU-Net_3tpt showed an intraclass correlation coefficient of 0.95 with the manually segmented reference at baseline -- meaning the automated tool's tumor size measurements were nearly identical to expert measurements. This correlation was 0.88 at C2 and 0.73 at C4, remaining clinically useful across the full treatment course. Accurate size measurement across timepoints is essential for quantifying treatment response -- tracking whether the tumor is shrinking, stable, or growing in response to chemotherapy.

TL;DR: Simple subtraction images outperformed complex parametric maps, and training across all three treatment timepoints achieved DSC 93% and sensitivity 96% -- matching expert segmentation with excellent tumor size correlation (ICC 0.95 at baseline).
Pages 4-5
How nnU-Net Configures Itself

Standard deep learning segmentation requires time-consuming expert configuration: choosing the right network architecture depth, image preprocessing steps, batch size, and training schedule typically requires weeks of experimentation and deep technical expertise. nnU-Net addresses this barrier through an automated pipeline that configures itself based on the dataset properties it receives.

When given a training dataset, nnU-Net first analyzes a data fingerprint -- the median image shape, intensity distribution, voxel spacing, and imaging modality. It then uses a dictionary of rules derived from benchmarking across the ten diverse datasets of the Medical Segmentation Decathlon to automatically set preprocessing parameters (resampling strategy, intensity normalization), batch size, and network topology. Fixed parameters like learning rate, optimizer type, and loss function are pre-determined from prior empirical optimization and remain unchanged regardless of the new application.

nnU-Net trains both 2D and 3D network variants in parallel, and at inference time combines their outputs through a majority vote on a voxel-by-voxel basis. The final prediction is an ensemble of models from five-fold cross-validation, which averages the output probability scores from five independently trained networks. Ensembling reduces prediction variance and typically yields higher accuracy and stability than any individual model alone.

This self-configuring design makes nnU-Net highly adaptable to new clinical applications without requiring a deep learning expert to tune each parameter. For clinical researchers who need a reliable segmentation tool but lack extensive machine learning infrastructure, nnU-Net's automated configuration removes the most significant technical barrier to deploying deep learning segmentation in a new disease context -- as this TNBC application demonstrates.

TL;DR: nnU-Net automatically configures its own preprocessing, architecture, and training strategy by analyzing dataset properties, then ensembles 2D and 3D models from five-fold cross-validation -- removing the need for manual hyperparameter tuning.
Pages 7-8
Necrosis, Biopsy Clips, and Model Flexibility

A practical question in breast tumor segmentation is whether to include or exclude central tumor necrosis and biopsy clips (small metal markers placed in the tumor during biopsy) within the segmented tumor volume. Necrotic tissue and clips appear as signal voids (dark regions) on MRI and are not biologically active tumor, so excluding them produces a more accurate measurement of viable tumor burden. But including them may be appropriate for other purposes, such as tracking the overall tumor footprint.

The study compared two segmentation models -- one trained to exclude necrosis and clips (nnU-Net_Excl) and one trained to include them (nnU-Net_Incl) -- and found no statistically significant difference in DSC between the two approaches. Both models produced accurate, comparable segmentations. This means the same nnU-Net framework can directly output either type of mask depending on what the clinical application requires, without any post-processing modifications.

This flexibility is clinically meaningful. Studies measuring functional tumor volume (the volume of metabolically active, treatment-sensitive tissue) need to exclude necrosis and clips to avoid overestimating viable disease. Studies measuring the radiological size of the treatment target may prefer to include these structures. Prior approaches required separate processing pipelines for the two mask types; this framework handles both in a single, unified model design.

TL;DR: nnU-Net performed equally well whether trained to include or exclude biopsy clips and necrosis from tumor segmentations, giving clinicians flexibility to choose the mask type most relevant for their application.
Pages 10-11
Comparison to Prior Work and Remaining Challenges

Compared to previously published breast cancer segmentation studies, nnU-Net_3tpt achieved superior or comparable DSC (0.93) despite the additional complexity of working exclusively with TNBC -- a subtype with particularly heterogeneous tumor morphology. Prior studies that achieved DSC of 0.87-0.89 used datasets of 800-1000 patients, while this study effectively leveraged 744 datasets by combining three treatment timepoints from the same patients. This multi-timepoint pooling strategy is a novel methodological contribution.

The key practical limitation is performance on small tumors. The model missed tumors smaller than 2 cm in several instances. As patients respond to neoadjuvant therapy, tumors shrink progressively, and by C4 many tumors fall below the threshold where the model performs reliably. This is the most critical failure mode for clinical translation, because C4 imaging is exactly the timepoint when detecting residual disease is most clinically important -- it directly informs whether surgery should proceed and what type of operation is most appropriate.

The authors propose several technical paths to address this. Replacing the standard DSC training loss with a focal Dice loss function could make the model penalize false negatives more heavily, reducing the rate of completely missed small tumors. Alternatively, incorporating semantic spatial features -- information about the three-dimensional context around each voxel -- could help the model detect faint enhancement patterns in small residual lesions that would otherwise be ignored as background noise.

The dataset used here came entirely from a single institution with a standardized imaging protocol. While this provides internal consistency, it means performance on data from other hospitals with different scanner types, coil configurations, or contrast injection protocols has not been tested. Multi-institutional validation is essential before this tool can be recommended for adoption at other cancer centers.

TL;DR: nnU-Net_3tpt matches or exceeds prior state-of-the-art segmentation accuracy, but the persistent challenge of small tumor detection (especially at later treatment timepoints) remains the primary barrier to full clinical utility.
Page 11
A High-Performance, Clinically Relevant Segmentation Tool

This study developed a fully automated, high-performance model for TNBC tumor segmentation on DCE-MRI images, achieving a Dice similarity coefficient of 93% and sensitivity of 96% using the nnU-Net framework trained across multiple treatment timepoints. The model accurately estimated tumor size (intraclass correlation coefficient 0.95 at baseline) and was flexible enough to produce both necrosis-inclusive and necrosis-exclusive segmentations from the same framework.

The finding that simple subtraction images outperformed more complex parametric maps as model input is practically significant: subtraction images are easy to generate without specialized software, making the pipeline deployable in most DCE-MRI environments without additional infrastructure investment. The multi-timepoint training strategy that pooled BL, C2, and C4 data into a single training set substantially improved performance over any single-timepoint approach, offering a template for how longitudinal imaging datasets can be leveraged to train more robust segmentation models.

Future work should focus on improving detection of small residual tumors at later treatment timepoints through modified loss functions and multi-scale feature extraction, validating performance on multi-institutional external datasets, and integrating the segmentation output with downstream quantitative analysis pipelines for radiomic feature extraction and treatment response prediction. This automated segmentation tool represents a foundation for AI-driven quantitative MRI analysis in TNBC clinical trials and routine care.

TL;DR: The nnU-Net_3tpt model achieves DSC 93% and sensitivity 96% for TNBC segmentation using simple subtraction DCE-MRI images, with accurate tumor size measurements -- providing a clinically deployable foundation for automated treatment response monitoring.
Citation: Open Access, 2023. Available at: PMC10571741.