Accurate histopathological classification of lung and colon cancer is among the most consequential and technically demanding tasks in clinical pathology. Pathologists examine cellular architecture, nuclear atypia, and mitotic patterns in tissue slides to distinguish cancer subtypes, but inter-observer variability and high slide volume make manual review error-prone and slow.
Deep learning models applied to histopathology images have pushed classification accuracy to exceptional levels, but single-model approaches relying solely on CNN-derived deep features have demonstrated poor generalization across datasets, overfitting to training data distributions and degrading when encountering new staining protocols or institutional slide variations.
Classical handcrafted features, including texture descriptors like Local Binary Pattern (LBP) and Gray Level Co-occurrence Matrix (GLCM), color statistics, wavelet decompositions, and morphological shape descriptors, capture domain-specific biological information that deep networks often do not learn explicitly, particularly structural and geometric properties of cellular organization.
This study proposes HandEffTrans, a hybrid framework that fuses classical handcrafted features with deep features from a modified EfficientNetB0 backbone using a transformer-based attention fusion mechanism, combined with adaptive incremental learning to support staged model training on accumulating clinical data without catastrophic forgetting.
The LC25000 dataset, containing 25,000 high-resolution H&E-stained histopathology images across five balanced classes, served as the primary training and evaluation dataset. The five classes are colon adenocarcinoma, normal colon, lung adenocarcinoma, normal lung, and lung squamous cell carcinoma, each containing 5,000 images selected for quality and clinical relevance.
To simulate real-world data accumulation and enable incremental learning, each class's 5,000 images were divided into five stages of 1,000 images each. The first stage (HandEffTrans-0) established a foundational model, and each subsequent stage (HandEffTrans-1 through HandEffTrans-5) introduced an additional batch of 1,000 images per class, updating model parameters while preserving previously learned representations.
Two independent external validation datasets were used to test generalizability: NCT-CRC-HE-100K, a colorectal cancer histopathology dataset with nine tissue classes from a different institution, and HMU-GC-HE-30K, a gastric cancer dataset with five tissue categories. These datasets introduced real-world variation in tissue type, staining protocols, and scanner characteristics.
5-fold cross-validation was applied throughout training, partitioning each incremental stage into five subsets and rotating the validation fold systematically, ensuring that every image contributed to both training and validation across the evaluation cycle. Performance was assessed by accuracy, precision, recall, F1-score, Cohen's Kappa score, and ROC curve analysis.
Classical feature extraction generates four complementary descriptor types from each histopathology image. Texture features are derived using Local Binary Pattern (LBP), which encodes local pixel intensity relationships to capture tissue microstructure, and Gray Level Co-occurrence Matrix (GLCM), which computes second-order statistical properties including contrast, dissimilarity, energy, and correlation across multiple orientations.
Color features are extracted by decomposing images into RGB and HSV color spaces and computing mean, standard deviation, maximum, and minimum values for each channel. Color variation is highly informative in H&E staining, where different tissue components absorb hematoxylin and eosin differentially, making color distribution a reliable discriminator between tissue types.
Wavelet features are extracted using Discrete Wavelet Transform, capturing both spatial and frequency information simultaneously. This multi-resolution analysis captures local fine-grained texture at high frequencies and global structural patterns at lower frequencies, complementing the purely spatial descriptors of LBP and GLCM.
Morphological features including cell area, eccentricity, solidity, and extent provide geometric and structural information about cellular organization. These features quantify nuclear shape irregularity and packing density that are directly relevant to histopathological grading of cancer aggressiveness and were not incorporated in prior hybrid approaches.
A modified EfficientNetB0 serves as the deep feature extraction backbone, initialized with ImageNet pretrained weights and fine-tuned for histopathological image classification. The base layers are frozen to preserve learned low-level representations from natural images while the top layers adapt to the domain-specific patterns of lung and colon tissue pathology. The final layer uses softmax activation for five-class classification.
EfficientNetB0 uses compound scaling to balance network depth, width, and resolution simultaneously, achieving high representational capacity with fewer parameters than comparable architectures such as ResNet50 or VGG. Its MBConv blocks with depthwise separable convolutions and Swish activation minimize computational cost while maintaining feature extraction quality.
The transformer-based attention fusion layer receives the concatenated classical and deep feature vectors and learns the inter-feature importance dynamically across the full feature set. Unlike static concatenation that treats all features equally, the attention mechanism identifies which classical descriptors and which deep feature dimensions are most relevant for each classification decision, producing an enriched fused representation.
Catastrophic forgetting during incremental training is mitigated by the adaptive attention fusion mechanism, which selectively retains useful features from previous training stages, combined with dynamic learning rate scheduling that reduces the learning rate in later incremental stages to preserve earlier knowledge while incorporating new class data. No traditional regularization techniques such as Elastic Weight Consolidation were required.
After five incremental training stages on the LC25000 dataset, the HandEffTrans-5 model achieved an overall accuracy of 99.87% with precision and recall of approximately 99.9% and a Cohen's Kappa score of 0.9983. This represents a clear progression from the initial HandEffTrans-0 baseline (98.4% accuracy) through each incremental stage, demonstrating that staged training consistently improved model performance without degradation from forgetting.
Performance on the external NCT-CRC-HE-100K colorectal histopathology dataset reached 99.07% accuracy and a Kappa score of 0.9883, with all five fold test accuracies remaining above 99% and Kappa values above 0.98. These results confirm that the fusion framework generalizes beyond the training tissue type and staining conditions without retraining from scratch.
On the HMU-GC-HE-30K gastric cancer dataset, accuracy reached 98.4% with a Kappa of 0.98, and fold-wise test accuracies remained stable above 98%. Minimal misclassifications appeared in tumor-stroma and debris-complex regions, where overlapping morphological features create inherent ambiguity even for expert pathologists.
Across all three datasets, precision, recall, and F1-scores exceeded 98%, and AUC values from ROC curves were consistently high for all classes, confirming strong discriminative ability across tissue categories ranging from colon adenocarcinoma to gastric lymphocyte and stroma classes despite substantial differences in tissue type, cancer origin, and imaging acquisition.
Comparative analysis against eight prior state-of-the-art methods on the LC25000 dataset showed that HandEffTrans-5 achieved the highest accuracy (99.87%) and perfect precision (100%), outperforming models such as Kadirappa et al. DeepHistoNet (99.80%) and ensemble methods using MobileNet and InceptionV3 (99.44%). Critically, prior methods were benchmarked only on LC25000, while HandEffTrans was additionally validated across two external datasets, providing substantially stronger evidence of real-world generalizability.
The incremental learning strategy directly addresses a key limitation of prior approaches: training neural networks on large static datasets does not reflect the reality of clinical deployment, where new patient data accumulates over time and models must adapt without reprocessing all historical data. HandEffTrans demonstrates that this can be achieved without sacrificing accuracy.
The inclusion of morphological features distinguishes this framework from previous hybrid approaches, which typically combined texture and color descriptors with deep features but omitted geometric cellular organization metrics. Morphological features such as eccentricity and solidity directly quantify pathological nuclear irregularities that human pathologists use for grading.
Limitations acknowledged by the authors include the lack of model compression for edge deployment in resource-constrained settings. Future directions include quantization and knowledge distillation to reduce inference time and memory requirements while preserving the classification accuracy demonstrated in this study, which would be necessary before clinical integration in low-resource pathology environments.
The proposed framework directly targets the two most critical barriers to clinical adoption of AI in histopathology: overfitting to training distributions and inability to adapt to new data. By achieving above 98% accuracy on three structurally different datasets and demonstrating that incremental staged training preserves knowledge across updates, HandEffTrans provides a foundation for sustainable clinical deployment.
Whole-slide image analysis represents the primary intended clinical use case, where automated scanning of tissue sections containing hundreds of cells across thousands of pixels per slide requires computational efficiency alongside high classification accuracy. The EfficientNetB0 backbone's parameter-efficient architecture supports this deployment context.
The framework's multi-class design, distinguishing lung adenocarcinoma, lung squamous cell carcinoma, and normal lung in addition to colon subtypes, is directly aligned with clinical practice where accurate subtype identification drives treatment selection decisions. Adenocarcinoma and squamous cell carcinoma have distinct driver mutations and targeted therapy options, making correct subtype identification clinically consequential.
Public availability of all code and training datasets ensures reproducibility and enables independent external validation by other research groups. This transparency supports the regulatory pathway for AI diagnostic tools and allows community extension of the framework to additional cancer types beyond lung and colon.
HandEffTrans demonstrates that fusing classical handcrafted features with deep learning representations through a transformer attention mechanism, combined with incremental learning, produces histopathology classifiers that generalize robustly across different cancer types, institutions, and tissue acquisition conditions. The framework challenges the assumption that accuracy gains require ever-larger models or ever-larger single training datasets.
The consistent improvement through incremental training stages, from 98.4% at initialization to 99.87% at stage five, validates that staged data introduction with adaptive attention fusion can effectively mimic the gradual accumulation of clinical experience that characterizes expert pathologist development, and can do so without performance regression on previously learned classes.
The external validation results, maintaining above 98% accuracy on colorectal and gastric cancer datasets despite the primary training on lung and colon tissue, suggest that the feature representations learned by the hybrid framework encode tissue-level biological features that transfer meaningfully across anatomical sites, an important finding for prospective multi-cancer deployment.
Future work should extend the framework to whole-slide image processing pipelines, include additional cancer types such as breast and prostate histopathology, incorporate model compression for resource-constrained clinical settings, and pursue prospective clinical validation to establish diagnostic equivalence with expert pathologists for regulatory approval.