Histopathological image analysis -- examining tissue samples under a microscope -- is the gold standard for cancer diagnosis. However, expert pathologists can disagree or make errors. Studies show that expert agreement rates for prostate cancer grading can be as low as 57.9%, and breast cancer subtyping accuracy can fall to 57%.
Artificial intelligence offers a path to more consistent and accurate analysis. Convolutional neural networks (CNNs) have transformed this field by automatically extracting meaningful patterns from raw tissue images without requiring manual feature engineering.
A major challenge is that most existing AI models are trained on specific datasets and struggle to generalize to new data from different hospitals or preparation methods. They also tend to focus on binary (cancer vs. no cancer) tasks rather than the clinically important multiclass categorizations that pathologists need.
Another problem is handling imbalanced datasets, where some cancer subtypes are far less common than others. Standard training methods become biased toward the majority class, reducing performance on the rarer but clinically important categories.
The researchers developed HistopathAI, a novel framework that combines two key innovations: Supervised Contrastive Learning (SCL) and Hybrid Deep Feature Fusion (HDFF). The core idea is that better features lead to better classifiers.
SCL works by pulling together images of the same cancer type in a mathematical space (the embedding space) while pushing apart images of different types. This produces tighter, more distinguishable clusters than standard loss functions and is especially helpful when training data is imbalanced.
HDFF combines feature vectors from two separate neural network architectures -- EfficientNetB3 and ResNet50 -- into a single 3,584-dimensional composite vector. EfficientNetB3 uses systematic scaling of network depth, width, and resolution, while ResNet50 uses skip connections to capture complex patterns at many levels.
The training proceeds in two stages. In Stage 1, both networks are pretrained together using SCL loss to learn rich, class-discriminative features. In Stage 2, the encoder weights are frozen and a classifier is trained on top of those features using standard cross-entropy loss. This stepwise approach mirrors the foundational concept of contrastive learning.
Supervised Contrastive Learning (SCL) addresses a fundamental weakness of standard cross-entropy (CE) loss functions. CE training can produce skewed, biased features that cluster poorly. SCL instead forces same-class samples to cluster tightly together and different-class samples to stay far apart in the embedding space.
Unlike self-supervised contrastive methods, SCL uses labeled data to define what counts as a 'positive' match (same class) and a 'negative' (different class). This avoids the problem of false negatives -- incorrectly treating a same-class image as a negative example -- which is especially harmful in cancer diagnosis settings.
The framework also uses probabilistic data augmentation during training, applying random transformations including rotations, flips, brightness changes, and cutout (randomly removing square regions) to diversify the training data and prevent the model from memorizing specific images.
Performance was evaluated using 5-fold cross-validation on eight datasets, and statistical significance was tested using the 5x2cv F-test -- a rigorous method for comparing classifiers. Metrics reported include precision, recall, F1 score, and overall accuracy.
HistopathAI was evaluated on seven publicly available datasets and one private in-house dataset, covering breast, colorectal, and gastric cancer across both binary (cancer vs. normal) and multiclass (multiple cancer subtypes) classification tasks.
Breast cancer datasets included BACH (4 classes: normal, benign, in situ carcinoma, invasive carcinoma), BRACS (7 classes ranging from normal tissue to invasive carcinoma), and BreaKHis (benign vs. malignant). Colorectal datasets included EBHI (5 classes from normal to adenocarcinoma) and MHIST (2 classes).
Gastric cancer was covered by GasHisSDB (33,284 images, normal vs. abnormal) and HE-GHI-DS. The in-house breast cancer dataset contained over 12,000 images of benign and malignant tissue, representing real-world conditions from a clinical pathology laboratory.
Images were standardized to 512x512 pixels and stored in an efficient binary format. Stratified sampling ensured equal class representation across cross-validation folds, which is particularly important for the imbalanced datasets used in this study.
HistopathAI -- the combined EfficientNetB3 + ResNet50 + SCL model -- consistently achieved the highest accuracy, precision, recall, and F1 scores across all eight datasets and classification tasks, outperforming all individual model baselines.
Performance highlights include: 99.59% accuracy on the BreaKHis binary breast cancer dataset, 99.17% accuracy on the GasHisSDB gastric cancer dataset, 99.43% accuracy on the HE-GHI-DS gastric dataset, and near-perfect (99.9%) accuracy on the in-house breast cancer dataset.
On more challenging multiclass tasks, HistopathAI still outperformed all alternatives. For example, on the 7-class BRACS breast cancer dataset, it achieved 78.9% accuracy compared to 78.1% for ResNet50 alone. The model's advantage was especially pronounced on the imbalanced GasHisSDB dataset, where adding SCL improved accuracy from 95.4% (EfficientNetB3 alone) to 99.2%.
Statistical testing using the 5x2cv F-test confirmed that HistopathAI's improvements were statistically significant in most comparisons, with F-statistics far exceeding the critical value of 3.2974 in the majority of cases.
HistopathAI's combination of SCL and HDFF addresses two persistent challenges in digital pathology: poor generalizability (models that work on one dataset but not another) and biased classification (models that struggle with rare cancer subtypes). The framework's design ensures it captures both high-level and low-level image features, producing more robust representations.
Visualization using t-SNE (a dimensionality reduction technique) confirmed that SCL produced clearly separated, well-defined clusters for each cancer class, while standard CE loss produced overlapping, intermingled clusters. This visual evidence reinforces the quantitative performance improvements.
Despite its strengths, the model has limitations. The 'black box' nature of deep learning makes it difficult for clinicians to understand why a decision was made -- an important barrier to clinical trust and adoption. Variability in staining techniques and slide preparation across different laboratories could also affect performance in real-world settings.
In clinical practice, HistopathAI could serve as a prescreening tool for pathologists: automatically triaging the most urgent or likely malignant cases for priority review, reducing visual fatigue, and enabling faster reporting in high-volume settings. Clinical adoption would require regulatory approval and rigorous local validation.
HistopathAI represents a meaningful advance in AI-assisted cancer diagnosis. By combining supervised contrastive learning with feature fusion from two complementary neural network architectures, it produces richer feature representations than any single approach alone.
The model's consistent superiority across eight diverse datasets -- from binary to 7-class classification, from breast to gastric cancer -- demonstrates that it generalizes well across different cancer types, image conditions, and class distributions.
The framework's test time is extremely low (milliseconds per image), making it practical for real-time clinical application even though training is computationally intensive. This balance of high accuracy and efficiency supports its potential use in digital pathology workflows.
Future work should explore integrating histopathological imaging with genomic data, improving model interpretability through explainability methods, and conducting prospective clinical validation studies to confirm real-world performance before widespread deployment.