When a doctor performs an endoscopic ultrasound-guided fine needle aspiration (EUS-FNA) to biopsy a suspicious pancreatic mass, the procedure is much more effective when a cytopathologist is present in the room to immediately check whether the sample contains enough cancer cells for diagnosis. This 'rapid on-site evaluation' (ROSE) increases diagnostic accuracy and reduces the need for repeat biopsies.
The problem is that cytopathologists are in short supply globally, and most hospitals — especially outside major medical centers — cannot staff them for every biopsy procedure. An AI system that could perform ROSE automatically, analyzing microscopy images of the biopsy material in real time, would bring this quality benefit to patients everywhere.
The ROSE-AI system was trained on whole-slide images of Diff-Quik stained EUS-FNA cytopathology slides from Okayama University Hospital. A total of 4,059 image patches from 36 pancreatic cancer patients and 9 non-cancer patients were used. The slides were digitized and divided into 256x256 pixel images for training.
The AI was built on a Vision Transformer (ViT) architecture — a state-of-the-art deep learning model that processes images as sequences of patches, similar to how language transformers process words. The model had 12 transformer layers, 86 million parameters, and was fine-tuned from ImageNet weights. Five-fold cross-validation was used to evaluate performance.
The central question of the study was: how much does data augmentation help? With only 45 patients' worth of data, the training set was small. Three augmentation strategies were compared: geometric transformations (rotations, flips, crops, Gaussian noise), color-space transformations (brightness, contrast, saturation changes), and kernel filtering (Gaussian blur). These were tested alone and in combination.
Without data augmentation, the ROSE-AI system achieved 87.5% sensitivity, 79.7% specificity, and 83.7% accuracy. The best augmentation strategy was geometric transformation alone, which improved accuracy to 88.2% — a modest but consistent improvement.
Surprisingly, color-space transformations and kernel filtering did not improve and in some cases decreased diagnostic ability. This suggests that artificial color changes confuse the model when the real diagnostic information in cytopathology slides lies in cell morphology (shape, size, arrangement) rather than color. Geometric transformations preserve cell morphology while teaching the model to be rotation-invariant, making them the appropriate augmentation for this task.
The model performed better on training images augmented with geometric transforms across all five cross-validation folds, confirming that this finding is robust and not a random result.
The Vision Transformer (ViT) model used in this study represents a newer approach to image analysis that differs from traditional convolutional neural networks (CNNs). Instead of learning local image features, ViT models process images as sequences of patches and use 'attention' mechanisms to learn global relationships — which may be better suited to identifying complex arrangements of cells in cytopathology.
Layer-wise relevance propagation (LRP) was used to visualize which parts of the slide images most influenced the AI's decisions, providing interpretability — an important feature for clinical acceptance. The whole-slide digitization and patch-based analysis pipeline was designed to be compatible with standard clinical workflows.
If deployed clinically, the ROSE-AI system would analyze biopsy slide images during the EUS-FNA procedure and immediately indicate whether cancer cells are present and whether the sample is adequate. This would allow the endoscopist to take additional needle passes if needed — before the patient leaves the procedure room.
The system could be particularly valuable in community hospitals without pathology support, in developing countries with limited specialist resources, and in high-volume endoscopy centers where pathologist time is limited. The relatively small training dataset (45 patients) used in this proof-of-concept study highlights the need for larger multi-center studies, but the results demonstrate the feasibility of the approach.
This study demonstrates that not all data augmentation is beneficial — the choice of augmentation technique must be matched to the specific characteristics of the medical images and the diagnostic task. For cytopathology, where cell morphology is the key diagnostic signal, geometric transforms are helpful while color transformations are not.
This insight has practical implications for researchers developing AI tools for other types of pathology images. The study also successfully demonstrated a prototype ROSE-AI system with clinically acceptable accuracy. Future work should expand training data through multi-center collaboration and test the system in a prospective clinical trial of real EUS-FNA procedures.