Improving ductal carcinoma in situ classification by CNN with exponential linear unit and rank-based weighted pooling

Complex Intell Systems 2021 Detection/Classification 7 Explanations View Original
Original Paper (PDF)

Unable to display PDF. Download it here or view on PMC.

Plain-English Explanations
Pages 1-2
Thermography as an Alternative DCIS Detection Tool

Ductal Carcinoma In Situ (DCIS) is a Stage 0 pre-cancerous lesion confined to breast milk ducts that has not invaded surrounding tissue. While highly treatable when caught early, it can progress to invasive breast cancer if left untreated. Early and accurate detection is therefore critically important for guiding appropriate intervention.

The standard imaging tool for DCIS detection is mammography, which uses ionizing X-ray radiation. Breast thermography (BT) offers a non-invasive alternative, using infrared cameras to capture heat distribution patterns across the breast surface. Because tumors stimulate blood vessel growth and increased blood flow, cancerous regions tend to emit more heat than surrounding healthy tissue, creating detectable asymmetric temperature signatures.

A key physiological principle behind thermography is that early-stage tumor cells produce nitric oxide, causing blood vessel widening (vasodilation) and subsequent angiogenesis, the formation of new blood vessels to feed the growing tumor. Both processes increase local blood flow and heat emission, which thermographic cameras can detect with temperature sensitivity as fine as 0.05 degrees Celsius.

While thermography has demonstrated roughly 88% accuracy in prior studies, its sensitivity can still be improved. Existing AI-based thermography systems relied on hand-engineered features such as texture descriptors and statistical measures, which are limited in their ability to capture subtle and locally complex patterns. This study aimed to improve performance by replacing manual feature engineering with an end-to-end deep learning approach.

TL;DR: Breast thermography detects the excess heat generated by tumors non-invasively, and this study uses deep learning to improve its accuracy for DCIS detection beyond the capabilities of prior AI methods.
Page 3
Dataset: Multi-Source Thermography Images

The dataset comprised 240 DCIS breast thermograms and 240 healthy breast thermograms assembled from five different sources: a prior study by the same group, two publicly available thermography databases, a mastology research database, and images collected via online search engines. This multi-source approach introduced variability in image characteristics that the model needed to handle.

All images were preprocessed to standardize them for model input. This involved cropping to remove background and isolate breast tissue, followed by resizing to a uniform 128 x 128 x 3 pixel format. Importantly, different image sources used different pseudo color maps to visualize temperature gradients, for example using yellow or red for high temperatures and blue or green for low temperatures. Rather than standardizing to one color scheme, the researchers intentionally preserved this variation to make the model robust to different display conventions.

Image labeling was performed by a consensus of expert radiologists. Two radiologists with over 10 years of experience independently reviewed each image. When they agreed, that label was assigned. When they disagreed, a third senior radiologist cast the deciding vote through majority consensus. This process ensured high-quality ground truth labels across the balanced two-class dataset.

TL;DR: 480 thermography images from five data sources were preprocessed and labeled by radiologist consensus to create a balanced DCIS versus healthy breast dataset for model training and testing.
Pages 3-5
Three Technical Innovations: ELU, RWP, and L-Way Augmentation

The study's CNN, called CNN-BDER, introduced three improvements over a standard convolutional neural network baseline. The first was replacing the traditional Rectified Linear Unit (ReLU) activation function with the Exponential Linear Unit (ELU). ReLU sets all negative neuron activations to zero, which can cause certain neurons to stop learning entirely. ELU instead allows small negative outputs for negative inputs, maintaining gradient flow and generally improving training stability and convergence speed.

The second innovation was replacing standard max pooling with rank-based weighted pooling (RWP). In a standard max pooling layer, only the single highest activation value in each small region is passed forward, discarding all other information. RWP instead assigns exponentially decreasing weights to values ranked from highest to lowest, so all values contribute to the output but higher-valued activations receive greater emphasis. This approach preserves more information from each feature map region while still emphasizing the most important activations.

The third contribution was a novel L-way data augmentation (LDA) strategy. From each training image, 16 different augmentation techniques were applied including gamma correction, rotation, scaling, horizontal and vertical shearing, random translation, color jittering across all three color channels, and Gaussian noise injection. The mirrored versions of each augmentation were also generated, resulting in approximately 480 new images derived from each original training image. This expanded the training set from 432 to over 200,000 images.

Five model variants were built progressively: Model-0 (base CNN), Model-1 (adding batch normalization and dropout), Model-2 (adding ELU), Model-3 (adding RWP), and Model-4 (adding both ELU and RWP). This systematic construction allowed the contribution of each component to be evaluated independently.

TL;DR: CNN-BDER introduces three specific technical improvements over standard CNNs: ELU activations, rank-based weighted pooling, and a 16-way data augmentation pipeline that inflates the training set by a factor of nearly 500.
Pages 10-11
Model Performance: Beating Prior AI and Human Experts

The full CNN-BDER model (Model-4 with LDA) achieved a sensitivity of 94.08%, specificity of 93.58%, and accuracy of 93.83% across 10 runs of 10-fold cross-validation. The reported standard deviations were low (below 2%), indicating that results were stable and reproducible across repeated testing runs. Additional metrics including F1 score, Matthews Correlation Coefficient, and Fowlkes-Mallows Index all confirmed consistent high performance.

Comparing the five models, each successive addition of batch normalization, dropout, ELU, and RWP contributed to performance gains. The full Model-4 outperformed all other variants for all seven performance indicators, and Kruskal-Wallis hypothesis testing confirmed that these improvements over Models 0, 1, 2, and 3 were statistically significant at the 5% level for most metrics.

The critical role of data augmentation was demonstrated by comparing all five models with and without LDA. Without LDA, even the best model achieved only 91.81% accuracy. The base models without LDA fell below 90% accuracy and underperformed compared to older non-deep-learning AI methods, illustrating that deep learning requires sufficient data volume to realize its full potential.

In comparison with eight prior state-of-the-art AI methods, CNN-BDER outperformed all of them. The next best competitor achieved 90.79% accuracy, while CNN-BDER reached 93.83%. Traditional methods such as naive Bayesian classifiers and support vector machines with handcrafted features lagged significantly further behind, with the worst achieving only 69.69% accuracy.

TL;DR: CNN-BDER achieved over 93% accuracy on DCIS thermography detection, statistically outperforming all four incremental model variants and eight previously published AI methods.
Page 13
Outperforming Human Radiologist Diagnosis

Three experienced radiologists were asked to independently classify all 480 thermography images as DCIS or healthy, having not previously seen any of the images. Their results ranged from 72.92% to 78.96% accuracy, substantially below the CNN-BDER model's 93.83%.

The best-performing radiologist (P3) achieved 82.50% specificity and 75.42% sensitivity. The weakest radiologist (P1) reached only 71.67% sensitivity and 74.17% specificity. These results demonstrate that visual interpretation of thermograms is inherently difficult for human observers, even experienced ones.

The approximately 15 percentage point gap between the AI model and the best human radiologist underscores the potential clinical value of automated thermography analysis. DCIS at Stage 0 presents subtle thermal changes that human observers may miss, while the CNN can systematically learn to detect these patterns through training on large augmented datasets.

TL;DR: All three expert radiologists achieved accuracy between 73% and 79% on the same thermography images, compared to 93.83% for CNN-BDER, suggesting substantial clinical value for AI-assisted thermography reading.
Pages 13-14
Why Deep Learning Outperforms Traditional Methods

The study demonstrates a clear hierarchy in classification performance: classical AI methods with handcrafted features performed worst, deep learning without data augmentation performed intermediately, and deep learning with LDA performed best. This pattern reflects a fundamental principle: CNNs learn richer hierarchical representations directly from image pixels than any manually designed feature extraction pipeline, but only when sufficient training data is available.

Earlier methods like wavelet energy entropy and fractal dimension capture global or coarse-grained texture information but struggle with the local, fine-scale patterns that distinguish DCIS from healthy tissue. The CNN's multiple stacked convolutional layers enable it to learn progressively more abstract representations, from low-level edge and texture detectors in early layers to high-level semantic patterns in deeper layers.

The L-way augmentation strategy is a critical enabler because the total dataset of 480 images is too small to train a deep network from scratch without overfitting. By generating approximately 480 synthetic variants per training image, the effective dataset size grows to over 200,000 examples, providing the diversity needed for robust learning. The augmentation techniques were chosen to simulate realistic variations in acquisition conditions, patient positioning, and camera calibration.

TL;DR: The performance gains from deep learning over traditional methods are explained by richer learned feature representations, while L-way data augmentation solves the small-dataset problem inherent in clinical imaging research.
Pages 13-14
Conclusions and Future Directions

CNN-BDER represents a meaningful advance in AI-assisted DCIS detection from thermography images, achieving 93.83% accuracy compared to 88% for prior thermography-based systems and roughly 73-79% for human radiologists on the same images. The three technical contributions, ELU activation, rank-based weighted pooling, and L-way data augmentation, each contributed independently to these performance gains.

The authors acknowledge several important limitations. The model has not been clinically validated in prospective settings or on independent patient cohorts. The model also currently operates only on thermography images and cannot process the more widely used mammography format. A clinically deployable system would ideally analyze both modalities to provide complementary diagnostic information.

Future work directions include expanding the thermography dataset, developing a hybrid model capable of processing both thermogram and mammogram inputs, and deploying the system as an online tool for radiologists worldwide to use and provide feedback on. If validated, such a system could serve as a low-cost, radiation-free second reader to flag suspicious cases for further clinical workup, potentially improving early DCIS detection rates in screening programs.

TL;DR: CNN-BDER achieves state-of-the-art DCIS detection from thermography images, though clinical validation and extension to mammography are needed before deployment as a practical screening aid.
Citation: Open Access, 2021. Available at: PMC8591711.