Histopathological examination of kidney tissue is the gold standard for renal cancer diagnosis, but manual microscopic evaluation is time-consuming, requires expert pathology expertise, and is subject to inter-observer variability. Automated computer-aided detection systems can provide consistent, scalable analysis of histological images.
Deep learning models such as AlexNet excel at learning high-level semantic features from images, but may not optimally capture fine-grained textural patterns that are diagnostically meaningful in histopathology. Traditional handcrafted texture descriptors like GLCM and Gabor filters specifically encode local spatial patterns that complement deep features.
This study investigated whether fusing deep CNN features extracted by AlexNet with five different handcrafted texture feature families from histopathology images improves renal cancer detection accuracy beyond what either feature type achieves alone, particularly in the context of a small hospital dataset.
The dataset consisted of histopathology images from 93 renal cancer patients and 150 healthy subjects collected between 2010 and 2019. After augmentation, the dataset was expanded to 630 renal cancer and 560 healthy images. Color normalization using SPCN (Structure-Preserving Color Normalization) was applied before feature extraction to standardize staining variations across slides.
Deep features were extracted from the fc7 fully connected layer of AlexNet, producing 4096-dimensional feature vectors per image. The top 100 most discriminative deep features were selected using statistical feature selection. Five texture feature families were then computed separately: HOG (Histogram of Oriented Gradients) and GLCM (Gray-Level Co-occurrence Matrix) from the statistical family; Gabor filter features from the transform-based family; LBP (Local Binary Pattern) from the statistical family; and MRF (Markov Random Field) features from the model-based family.
Fused feature vectors were created by concatenating the selected AlexNet deep features with each of the five texture feature sets individually, and with all five combined. Multiple classifiers were evaluated including SVM, KNN, and others. The best combination and classifier were identified by systematic comparison.
Performance was evaluated by classification accuracy on held-out test splits, with statistical significance assessed by comparison to baseline individual feature set performance.
AlexNet alone, without any texture augmentation, achieved a classification accuracy of 87.72% for renal cancer versus healthy tissue discrimination. This establishes a strong baseline that the fusion approaches must meaningfully exceed to justify the added complexity.
The best-performing combination was AlexNet + Gabor filter features classified by SVM, achieving 98.54% accuracy, a substantial improvement of nearly 11 percentage points over the deep features alone. This finding identifies Gabor filter features as the most complementary texture modality for AlexNet deep features in this histopathological context.
All five fusion approaches outperformed the individual component methods (AlexNet alone and each texture descriptor alone), confirming the consistent benefit of feature fusion across different texture modality choices. The statistical family methods (GLCM, LBP, HOG) also showed competitive accuracy when fused, though Gabor was the single best pairing.
Gabor filters are linear filters tuned to specific spatial frequencies and orientations, inspired by the response properties of visual cortex simple cells. Applied to histopathology images, they capture periodic texture patterns such as glandular or nuclear arrangements at multiple spatial scales and orientations simultaneously.
AlexNet's fc7 layer features capture high-level semantic shape and appearance patterns learned from large image databases. Gabor features, by contrast, encode local frequency-selective texture regularity that is not explicitly represented in the deep feature hierarchy. This complementarity at different levels of image abstraction explains why their fusion is particularly synergistic.
SPCN color normalization was critical for reliable texture feature extraction because handcrafted texture descriptors are computed on intensity or color channels that would vary substantially across different staining batches without normalization. Deep features are somewhat more robust to color variation due to learned invariances, making SPCN proportionally more important for the texture component of the fusion.
A key practical advantage of this approach is its applicability to conventional digital pathology images rather than requiring specialized whole-slide imaging (WSI) scanners. Many smaller hospitals and pathology departments in resource-limited settings capture histological images at standard microscope resolutions, and this method can be implemented on such images.
The relatively small training dataset of 243 patients expanded to 1,190 images through augmentation is realistic for a single institution. The strong performance (98.54%) achieved on this scale suggests that fusion-based approaches can be effective even without the massive datasets typically required to train deep learning models from scratch.
Automated histopathology screening could serve as a second reader to flag potentially cancerous slides for pathologist review, reducing the number of slides requiring manual examination and prioritizing the most urgent cases. In high-volume pathology laboratories, this could meaningfully reduce turnaround time for renal cancer diagnoses.
This study provides systematic empirical evidence that fusing deep CNN features with handcrafted texture descriptors consistently improves renal cancer detection accuracy across multiple texture modality choices, with the Gabor filter providing the greatest complementary benefit to AlexNet features.
The 98.54% accuracy achieved by the best fusion approach represents a clinically relevant performance level for a second-reader screening application, though prospective clinical validation with independent patient cohorts from different institutions would be required before deployment in actual diagnostic workflows.
Future work should explore fusion of deep features from more recent architectures such as ResNet or Vision Transformers, investigate multi-scale texture analysis, extend the approach to multi-class subtype classification (ccRCC versus pRCC versus chRCC), and validate performance on genuinely independent external cohorts to assess geographic and scanner-related generalizability.