When a suspicious pancreatic mass is biopsied, pathologists examine the tissue under a microscope to determine whether it is cancer. This process — called histopathological analysis — requires expert training and significant time, and results can vary between pathologists.
Deep learning models have shown promise in automating aspects of this analysis, but their reliability across different hospitals and imaging systems is often untested. A model trained at one institution may perform poorly at another due to differences in how tissue is prepared, stained, and digitized.
Understanding how AI models perform across multiple real-world datasets — and how to make them more generalizable — is essential before they can be safely deployed in clinical pathology laboratories.
Researchers trained and tested weakly-supervised deep learning models using two major publicly available datasets of pancreatic ductal adenocarcinoma (PDAC) tissue images: The Cancer Genome Atlas (TCGA) and the Clinical Proteomic Tumor Analysis Consortium (CPTAC).
The term 'weakly supervised' means the AI was only told whether each whole slide image was cancer or normal — not which specific regions of the slide contained cancer cells. This is more realistic than approaches requiring detailed pixel-by-pixel labeling by pathologists.
Because TCGA alone lacked enough normal (non-cancerous) tissue samples for training, researchers also integrated data from the GTEx project, which provides healthy tissue samples. Models were then tested both within and across datasets to measure generalizability.
The model trained on CPTAC generalized better to new data, achieving an accuracy of 90.62% when tested on the TCGA+GTEx dataset, and 92.17% when tested on an external tissue microarray dataset. However, the model trained on TCGA+GTEx did not perform as well when tested on CPTAC images.
Analysis revealed a concerning phenomenon: when using an integrated dataset of images from different sources, the AI learned to distinguish between the datasets themselves rather than between cancer and normal tissue. This is called 'batch effect' — a form of bias caused by technical differences between data sources.
When all three datasets were combined for training, the batch effect was reduced and model performance improved across all test sets. Training on more diverse data helped the AI learn genuinely cancer-relevant features rather than dataset-specific artifacts.
Batch effect is a well-known problem in computational biology: when data comes from multiple sources with different technical processes, statistical differences between sources can dominate over the biological signal of interest.
In digital pathology, differences in tissue preparation protocols, staining intensity, slide scanning hardware, and digital image processing all create subtle but consistent patterns that an AI can detect. If the training dataset has cancer images from hospital A and normal images from hospital B, the AI may simply learn 'hospital A = cancer.'
Normalization techniques — computational methods that standardize the appearance of images across sources — are one approach to reducing batch effect, but the results here show that dataset composition (ensuring both cancer and normal examples come from the same sources) may be even more important.
This study provides important practical guidance for building reliable AI systems for cancer pathology. Simply having a large dataset is not enough — the composition of the dataset, and how different data sources are balanced, profoundly affects whether the model learns to detect cancer or learns to identify which hospital the data came from.
The finding that combining all three datasets improved generalizability points to the value of large, diverse training sets that span multiple institutions, imaging systems, and patient populations. Clinical AI tools should be built and validated with this diversity in mind.
For pancreatic cancer specifically, where early accurate diagnosis is critical, deploying an AI pathology tool without rigorous multi-center validation could give false confidence. This research helps define the standards that should be applied before any such system is used in clinical care.