Classification of Lymphoma Pathological Images Based on Deep Residual Neural Networks

Technology and Health Care 2021 AI 8 Explanations View Original
Original Paper (PDF)

Unable to display PDF. Download it here or view on PMC.

Plain-English Explanations
Pages 1-2
Why Automated Lymphoma Classification Is Clinically Urgent

Malignant lymphoma is a tumor of the lymphohematopoietic system accounting for approximately 50% of all malignant blood diseases. The World Health Organization (WHO) classification recognizes nearly 100 distinct subtypes, creating a diagnostic landscape of exceptional complexity. Among the most clinically common and diagnostically challenging subtypes are chronic lymphocytic leukemia (CLL), follicular lymphoma (FL), and mantle cell lymphoma (MCL) - three B-cell neoplasms that share overlapping morphological features on routine histopathology sections but carry markedly different prognoses and treatment requirements.

The rising disease burden: According to the National Cancer Center of China's 2015 prevalence report, the mortality rate from malignant lymphoma in China was 2.39 per 100,000 population, placing it 10th among all malignant tumor deaths. Both incidence and mortality have trended upward in recent years, reinforcing the need for faster, more accurate diagnostic tools to support earlier intervention and appropriate treatment selection.

The diagnostic bottleneck: Pathological analysis of Hematoxylin-Eosin (H&E) stained tissue sections is the established gold standard for lymphoma subtyping. However, this approach has three well-recognized problems. First, there is a global shortage of expert hematopathologists, and diagnostic competence varies substantially between institutions, increasing workload for experienced specialists. Second, prolonged review of high-magnification microscopy images induces visual fatigue, contributing to misdiagnosis and missed diagnoses. Third, diagnostic conclusions depend on the subjective judgment of individual pathologists, and different reviewers may reach different conclusions from identical specimens - producing inconsistent, potentially incorrect, treatment guidance.

These limitations collectively motivate the development of automatic classification systems that can analyze pathological images with consistent, reproducible criteria. This paper presents a deep learning approach using the ResNet-50 residual network architecture to classify CLL, FL, and MCL from digitized pathology images, comparing its performance against two classical neural network baselines.

TL;DR: Lymphoma comprises nearly 100 WHO-defined subtypes and accounts for ~50% of malignant blood diseases. Mortality in China is 2.39 per 100,000. Traditional H&E-based diagnosis is limited by specialist shortages, visual fatigue, and interobserver variability. This paper uses ResNet-50 to automate classification of CLL, FL, and MCL - the three most common and morphologically overlapping subtypes.
Pages 2-3
A Decade of Progress in Automated Lymphoma Image Classification

Research on automated lymphoma image analysis has progressed substantially since the late 1990s, evolving from rule-based image processing and classical machine learning toward deep convolutional neural networks. Understanding this trajectory is important for contextualizing the contribution of the ResNet-50 approach proposed in this paper.

Classical machine learning era: Early work by Ruschenburg et al. (1999) applied classification trees from decision tree methods to distinguish low-grade NHL from myoepithelial sialadenitis. By 2010, Orlov et al. leveraged global image features - shape descriptors, texture statistics, and transform-based representations - to classify FL, CLL, and MCL using the WND-CHARM classifier, establishing one of the first publicly available lymphoma image benchmarks. In 2011, Zorman et al. used symbol-based machine learning on follicular lymphoma images, and by 2015, Ting et al. applied an improved support vector machine (SVM) algorithm to classify seven lymphoma subtypes simultaneously, demonstrating that kernel-based classifiers could achieve meaningful accuracy across a broader subtype range.

Transition to deep learning: The shift toward convolutional neural networks (CNNs) accelerated after 2018. Guan et al. (2019) applied the Inception-V3 architecture to classify NHL and squamous cell carcinoma from cytological images, achieving an overall accuracy of 89.62%. Brousset et al. (2019) compared VGGNet and GoogLeNet architectures for distinguishing FL from diffuse large B-cell lymphoma (DLBCL), demonstrating that even simple CNN configurations could usefully assist pathologists. More recently, Biaosheng et al. (2020) deployed Faster R-CNN - an object detection framework - to recognize and classify lymphoma pathological images, reporting 96% classification accuracy and advancing the field toward localization-based approaches that identify and classify specific regions of interest within slides.

The consistent limitation across these prior studies is reliance on manually engineered feature extraction pipelines, shallow architectures prone to gradient-related training instabilities, or single-institution datasets that do not reflect the diversity of clinical pathology material. ResNet-50's skip-connection design directly addresses the gradient degradation problem that constrained earlier deep network approaches.

TL;DR: Prior approaches evolved from decision trees (1999) through WND-CHARM global features (2010), SVM on 7 subtypes (2015), Inception-V3 at 89.62% accuracy (2019), VGGNet and GoogLeNet (2019), and Faster R-CNN at 96% (2020). Each generation improved on feature engineering limitations. ResNet-50 addresses the gradient degradation problem that constrained earlier deep network training.
Pages 3-4
Dataset, Preprocessing, and the Baseline Neural Networks

The experimental dataset consists of 374 digitized pathological images from the publicly available Kaggle lymphoma dataset originally described by Orlov et al. (2010), covering three lymphoma types: CLL, FL, and MCL. Each source image is 1388 x 1040 pixels and stored in TIF format. The dataset was partitioned into training, validation, and test sets in a 6:2:2 ratio - approximately 224 images for training, 75 for validation, and 75 for testing. The validation set was used for hyperparameter tuning and model selection, with the test set held out for final generalization evaluation.

Data augmentation: With only 374 total images, overfitting is a significant risk for any deep learning model. The authors addressed this using three image transformation techniques. Image flipping (horizontal and vertical inversion) directly doubles or quadruples the number of training examples by exploiting rotational invariance in tissue sections - pathology images have no fixed orientation. Affine transformation introduces controlled geometric distortions: the mapping is a linear transformation from one 2D plane to another, parameterized by a 2x3 coefficient matrix, producing rotations, shears, and translations that simulate variation in tissue preparation and mounting. Size adjustment standardizes all input images to 512 x 512 pixels as required by the network input layer. These augmentation steps collectively expand the effective training set and improve model generalization across tissue preparation variability.

BP neural network baseline: The backpropagation (BP) neural network proposed by Rumelhart and McCelland (1986) served as the first comparison model. It consists of an input layer, one or more hidden layers, and an output layer, trained via gradient descent with error backpropagation. While BP networks can learn nonlinear decision boundaries, they are limited by shallow architectures that cannot extract the hierarchical feature representations required for complex image classification. The BP baseline achieved 96% accuracy on the lymphoma dataset at 512 x 512 input resolution.

GA-BP neural network baseline: The second comparison model augments BP with a genetic algorithm (GA) optimization layer. Because BP optimization by gradient descent is a local search method prone to getting trapped in local minima - particularly in the complex, high-dimensional loss landscapes of pathology image data - the GA provides a global search capability that simultaneously evaluates multiple candidate weight configurations (individuals in a population). The GA-BP network improves initial weight initialization and bias parameters before fine-tuning with standard backpropagation, resulting in faster convergence and reduced risk of suboptimal local solutions. GA-BP achieved 97.7% accuracy on the same dataset.

TL;DR: Dataset: 374 CLL/FL/MCL images from Kaggle (1388x1040 px TIF), split 60/20/20. Augmentation via flipping, affine transformation, and resizing to 512x512. BP baseline achieved 96% accuracy; GA-BP (genetic algorithm optimization of weights) achieved 97.7%. Both are compared against ResNet-50 on identical data splits.
Pages 4-5
ResNet-50: Residual Connections and the Solution to Gradient Degradation

ResNet-50, proposed by He et al. (2016) and awarded the IEEE CVPR Best Paper, was selected as the primary architecture for this study. Its defining innovation is the residual learning framework, which reformulates what each network layer learns. Rather than directly learning a desired mapping H(x) from input to output, each residual block learns the residual function F(x) = H(x) - x, and the block output is computed as x(i+1) = F(x) + x. The added "x" term is a shortcut connection that bypasses one or more convolutional layers and directly adds the input to the transformed output.

Why skip connections matter: As neural networks deepen, gradient signals propagating backward through many layers during training become increasingly small (gradient vanishing) or unstable (gradient explosion), making training difficult or impossible. In a standard deep network without skip connections, F(x) is pushed toward zero as the network deepens - effectively causing layers to degrade toward identity mappings, which wastes representational capacity. With skip connections, even if F(x) tends toward zero, the output still carries the original input x through the identity mapping, preserving gradient flow and enabling stable optimization across all 50 layers. Crucially, skip connections add no new parameters and do not increase computational complexity, so the benefits come at no architectural cost.

ResNet-50 structure: The network contains 50 layers organized into one input convolutional layer, four residual stages, and a fully connected classification layer followed by a softmax output. The four residual stages contain 3, 4, 6, and 3 bottleneck blocks respectively, each bottleneck consisting of three convolutional layers (1x1 to reduce dimensionality, 3x3 for spatial feature learning, 1x1 to restore dimensionality), totaling 48 convolutional layers within the residual stages. Channel widths progress through 64, 128, 256, and 512 feature maps across the four stages, enabling progressively abstract feature representations from low-level edge and texture detection to high-level morphological pattern recognition.

Training configuration: The model was trained using the cross-entropy loss function, which is standard for multi-class classification and measures the divergence between predicted class probabilities and one-hot ground truth labels. Each convolutional layer is followed by Batch Normalization (BN), which normalizes activations across the mini-batch to stabilize and accelerate training convergence. The ReLU activation function introduces nonlinearity at each layer. Key hyperparameters: learning rate of 0.0001, batch size of 32, and 100 training epochs with 3 output classes. Training was conducted on Intel Xeon Gold 5220 CPU and GeForce RTX 2080 Ti GPU using the PyTorch framework with Python.

TL;DR: ResNet-50 uses skip connections (x(i+1) = F(x) + x) to prevent gradient vanishing across 50 layers - 48 convolutional layers in 4 residual stages plus input and fully connected layers. BN after every conv layer accelerates convergence. Training: cross-entropy loss, lr=0.0001, batch=32, 100 epochs, PyTorch on RTX 2080 Ti GPU.
Pages 5-6
Classification Accuracy and Statistical Comparison Against Baselines

After training and hyperparameter optimization, the ResNet-50 model was evaluated on the held-out test set using the average accuracy rate (ACC) as the primary metric. Accuracy was computed as ACC = (TP + TN) / (TP + FP + TN + FN), where TP (true positive) counts correctly predicted positive-class samples, TN (true negative) counts correctly predicted negative-class samples, FP (false positive) counts negative-class samples incorrectly predicted as positive, and FN (false negative) counts positive-class samples incorrectly predicted as negative. Since the task is multi-class, each class is treated as the positive class in turn, with the remaining classes serving as negative, and accuracy is averaged across all three class-specific evaluations.

Performance comparison: ResNet-50 achieved 98.63% accuracy on the 512x512 input images, compared to 96% for the standard BP neural network and 97.7% for the GA-BP optimized network. The improvement of ResNet-50 over BP (2.63 percentage points) and over GA-BP (0.93 percentage points) reflects the advantages of deep hierarchical feature learning over both shallow networks and classically optimized shallow networks. All three models were evaluated on identical data splits to ensure fair comparison.

Statistical significance: To confirm that performance differences were not attributable to random variation, the authors applied paired t-tests comparing ResNet-50 against each baseline. The ResNet-50 vs. BP comparison yielded a t-value of 8.281 with p-value less than 0.05, indicating a statistically significant difference. The ResNet-50 vs. GA-BP comparison yielded a t-value of 7.800, also with p-value less than 0.05. Both comparisons confirm that ResNet-50's accuracy advantage is unlikely to be due to chance, providing statistical evidence that the residual network architecture produces meaningfully better classification performance on this lymphoma dataset.

The results demonstrate that deep residual learning captures pathologically relevant image features - cellular morphology, nuclear shape, growth pattern, texture - more effectively than either backpropagation alone or backpropagation with genetic algorithm weight optimization. The 98.63% accuracy is particularly notable given the small dataset size of 374 total images, suggesting that the combination of residual learning and data augmentation was effective in extracting generalizable representations from limited training material.

TL;DR: ResNet-50: 98.63% accuracy. BP baseline: 96%. GA-BP baseline: 97.7%. Statistical significance confirmed by paired t-tests: ResNet-50 vs. BP t=8.281 (p<0.05), ResNet-50 vs. GA-BP t=7.800 (p<0.05). All models trained and tested on identical 374-image splits at 512x512 input resolution.
Pages 6-7
The Biology Behind CLL, FL, and MCL Classification Challenges

Understanding why automated classification is difficult requires examining the morphological characteristics of each lymphoma subtype and the features that distinguish them. CLL, FL, and MCL represent three distinct B-cell lymphoid neoplasms with partially overlapping histological appearances, particularly at low magnification, which is precisely the challenge the ResNet-50 model is trained to resolve.

Chronic lymphocytic leukemia (CLL): CLL is a clonal proliferation of B lymphocytes morphologically similar to normal small lymphocytes. On H&E sections, CLL lymph node involvement shows a diffuse architectural effacement by monotonous small lymphocytes with condensed nuclear chromatin and scant cytoplasm, often with scattered larger prolymphocytes and mitotic figures (proliferation centers or pseudofollicles). The relative uniformity of cell size and nuclear morphology, combined with the diffuse rather than nodular growth pattern, are the key morphological discriminators from the other two subtypes.

Follicular lymphoma (FL): FL derives from B cells of the follicular germinal center and is characterized by its nodular (follicular) architecture - back-to-back follicles replacing the normal lymph node structure. Cells include centrocytes (small, irregular, cleaved nuclei) and centroblasts (larger cells with vesicular nuclei and prominent nucleoli). The follicular growth pattern is the principal diagnostic feature visible at low magnification and is what CNN spatial feature detectors would be expected to learn. Distinguishing FL from reactive germinal centers (which also show follicular architecture) requires recognition of the absence of tingible body macrophages and polarization seen in reactive follicles.

Mantle cell lymphoma (MCL): MCL results from t(11;14) chromosomal translocation causing cyclin D1 overexpression, which drives aberrant cell cycle progression and continuous proliferation. The tumor consists of small to medium-sized B lymphocytes morphologically resembling centrocytes, with irregular nuclear contours (nuclear indentations and convolutions) and scant cytoplasm. MCL most commonly shows a mantle zone or diffuse growth pattern. The irregular nuclear morphology - particularly the characteristic nuclear "notching" and convolutions - is the key microscopic feature that distinguishes MCL from CLL. The fact that ResNet-50 can distinguish these three subtypes at 98.63% accuracy indicates that deep convolutional feature detectors can capture the architectural, cellular, and nuclear features that pathologists use diagnostically.

TL;DR: CLL shows diffuse monotonous small lymphocytes; FL shows nodular follicular architecture with centrocytes and centroblasts; MCL arises from t(11;14)/cyclin D1 overexpression and shows irregular nuclear morphology. Overlapping appearances at low magnification are why automated classification is challenging - and why ResNet-50's 98.63% accuracy across all three is clinically meaningful.
Pages 7-8
Constraints on Current Performance and Generalizability

Small dataset with limited subtype coverage: The most significant limitation is the 374-image dataset size. While data augmentation mitigated overfitting within the three target classes, 374 images is far smaller than the datasets used to train state-of-the-art computational pathology models. More critically, the model was trained and validated exclusively on CLL, FL, and MCL - three of the nearly 100 WHO-classified lymphoma subtypes. Clinical lymphoma pathology regularly encounters DLBCL (the most common lymphoma subtype), Hodgkin lymphoma, marginal zone lymphoma, peripheral T-cell lymphoma, and numerous rarer entities. A classifier trained on only three subtypes has no mechanism to reject images outside its training distribution, creating a practical deployment problem: the model cannot recognize cases where a fourth or fifth subtype is present.

Morphological overlap with similar entities: The authors acknowledge that classifying lymphoma subtypes with similar morphology but different treatment regimens and prognoses remains difficult. Within the three-class problem addressed, ResNet-50 achieved 98.63% accuracy, but the real clinical challenge is broader. Distinguishing MCL from marginal zone lymphoma, or low-grade FL from reactive follicular hyperplasia, involves morphological subtleties that require immunohistochemistry (IHC) panels and often molecular testing. The model addresses only the H&E image classification component and does not integrate IHC, flow cytometry, cytogenetics, or molecular data that would be required for confident real-world diagnosis.

Single public dataset and lack of external validation: All experiments used the Kaggle lymphoma dataset (originally from Orlov et al., 2010). This dataset was generated from a specific set of laboratories with particular staining protocols and scanning equipment. Performance on images from different institutions - with variation in H&E staining intensity, tissue fixation, slide thickness, and digital scanner characteristics - has not been tested. The absence of external validation on independent institutional datasets is a standard limitation of single-source computational pathology studies and limits the generalizability claims that can be made.

Limited explainability: The ResNet-50 model produces classification outputs without identifying which morphological features drove each prediction. For clinical adoption, pathologists need to understand whether the model is responding to diagnostically meaningful features (follicular architecture, nuclear morphology, growth pattern) or to spurious correlates of the training data. Visualization techniques such as Grad-CAM attention maps were not applied in this study, leaving interpretability as an unaddressed limitation.

TL;DR: Key limitations: (1) only 374 images from a single public dataset with no external validation; (2) classification covers only 3 of nearly 100 lymphoma subtypes with no out-of-distribution rejection; (3) model operates on H&E alone without IHC, cytogenetics, or molecular data; (4) no explainability analysis (Grad-CAM or equivalent) to confirm the model responds to diagnostically meaningful morphological features.
Pages 8-9
Expanding the Model Toward Clinically Complete Lymphoma Classification

Broader subtype coverage: The authors explicitly identify expanding the classification scope as the primary future direction. A clinically useful automated lymphoma classifier would need to cover at minimum the major WHO subtypes - DLBCL, FL, CLL, MCL, Hodgkin lymphoma, marginal zone lymphomas, T-cell lymphomas, and Burkitt lymphoma - and ideally more uncommon entities. This requires assembling substantially larger datasets, ideally through multi-institutional collaborations that pool cases from different pathology laboratories. Federated learning approaches, which enable model training across institutions without transferring patient data, represent a promising mechanism for building the large, diverse datasets needed without compromising data privacy.

Morphologically similar subtypes: The paper specifically notes the difficulty of classifying lymphoma subtypes with similar morphological appearances but different clinical trajectories. This challenge points toward integrating multiple data modalities beyond H&E images. IHC biomarker panels, flow cytometry immunophenotyping, fluorescence in situ hybridization (FISH) for chromosomal translocations (such as the t(11;14) defining MCL), and next-generation sequencing mutational profiles all contribute clinically essential information that H&E alone cannot provide. Multimodal deep learning architectures that jointly process imaging and molecular data would better reflect how expert hematopathologists actually reach diagnostic conclusions.

Transfer learning and foundation models: Training from scratch on 374 images - even with data augmentation - is not optimal. Transfer learning from large pre-trained pathology foundation models (such as those trained on millions of TCGA slides) or natural image models (ImageNet-pretrained ResNet weights) would leverage learned feature representations from vastly larger datasets. The paper used ResNet-50 within a single-institution training paradigm; applying it within a transfer learning framework, or fine-tuning a pathology-specific foundation model, would likely improve both accuracy and generalization with the same training data.

Clinical integration: The broader trajectory points toward integrating deep learning classification into digital pathology workflows, where whole-slide scanners digitize biopsy sections and AI models triage and classify cases before pathologist review. This paradigm - AI as a first-pass screening and classification tool rather than a replacement for pathologist judgment - is increasingly being demonstrated in other hematologic malignancy settings and represents the most pragmatic near-term deployment pathway for systems like the one described here.

TL;DR: Key future directions include expanding from 3 subtypes to broader WHO coverage via multi-institutional datasets; integrating IHC, FISH, and molecular data for morphologically overlapping subtypes; applying transfer learning from large pathology foundation models rather than training from scratch on 374 images; and deploying as a first-pass triage tool within digital pathology workflows.