Pancreatic neuroendocrine tumors (NETs) arise from islet cells of the pancreas and have a better prognosis than ductal adenocarcinoma, yet their clinical behavior is unpredictable. The Ki67 index, which reflects the fraction of tumor cells actively dividing, is the most reliable prognostic factor and is central to the WHO grading system for digestive system NETs.
The WHO mandates counting at least 500 Ki67-positive tumor cells in a hotspot region of 10 high-power fields to assign a tumor grade. Grade I requires a Ki67 index below 3%, Grade II between 3 and 20%, and Grade III above 20%. Accurate grading directly determines treatment decisions, yet the counting process is laborious and subject to inter-pathologist variability.
A fundamental challenge in Ki67 analysis is that the antibody stains both tumor and non-tumor cells that are dividing. Similarly, the counterstain (hematoxylin) labels both tumor and non-tumor nuclei regardless of division status. This non-specificity means that automated image analysis cannot reliably count Ki67-positive tumor cells without first identifying which cells are tumor.
Existing solutions to the tumor identification problem relied on expensive quantum dot double staining or image registration between adjacent Ki67 and pancytokeratin stained tissue sections. Both methods have practical limitations that prevent their routine clinical use, motivating the development of a simpler deep learning approach.
The study used 33 whole-slide Ki67-stained images from 33 patients with pancreatic NETs, digitized at 20x magnification using an Aperio ScanScope scanner at 0.2437 microns per pixel. Expert pathologist annotations identified tumor and non-tumor regions in each slide, yielding 129,024 tumor tiles and 9,032 non-tumor tiles of 64x64 pixels for training and validation.
Transfer learning was applied using the Inception-v3 convolutional neural network, which was originally trained on ImageNet to classify natural images into 1,000 categories. Rather than training the full network on the small pathology dataset, only the softmax classification layers were retrained on the two-class tumor versus non-tumor problem, using the pre-trained network as a feature extractor. This approach is particularly well-suited when labeled domain-specific data is limited.
As a baseline comparison, AlexNet was also evaluated using fine-tuning, in which error was allowed to backpropagate through the entire network. Training used leave-one-out cross-validation across all 33 whole-slide images, ensuring that the test slide was always excluded from training. A sliding window of 64x64 pixels with a step size of 8 pixels produced probability maps for tumor and non-tumor regions across each test image.
For evaluation, 30 high-power field (HPF) images were cropped from regions not used in training and assessed by two independent expert pathologists (B and C). Two consensus readings were defined: C1 (intersection of both pathologists' annotations) and C2 (union of both pathologists' annotations), providing conservative and permissive ground truth benchmarks respectively.
Against the C2 consensus reading (union of both pathologists' annotations), Inception-v3 achieved a sensitivity of 97.8% for tumor pixels and a specificity of 88.8% for non-tumor pixels. Overall accuracy was 96.2%. Against the stricter C1 consensus (intersection of both pathologists' annotations), accuracy reached 99.3%.
The two pathologists evaluating the test images had an inter-rater agreement of 96.2% for tumor regions but only 83.6% for non-tumor regions. The lower non-tumor agreement was attributed to the small and scattered nature of non-tumor cell clusters, which were more challenging for pathologists to annotate exhaustively and consistently.
Inception-v3 trained by transfer learning was substantially superior to AlexNet trained by fine-tuning across all consensus and bootstrapping conditions. The advantage of Inception-v3 was attributed to its architectural features, particularly inception modules that reduce overfitting through parallel multi-scale convolutions, and label smoothing regularization that prevents overconfident misclassification.
The average training and validation accuracy across all 33 leave-one-out models was 86.7% (standard deviation 0.82%), indicating consistent performance across the varied tissue appearances in the dataset without significant overfitting to any single slide.
Fine-tuning AlexNet by allowing error to propagate through all layers often causes overfitting when training data is limited. With only 33 whole-slide images and a highly imbalanced dataset (approximately 14 times more tumor tiles than non-tumor tiles), a fully fine-tuned network memorizes training examples rather than learning generalizable features.
Inception-v3's architectural innovations further reduced the risk of overfitting. Its inception modules perform parallel convolutions at multiple kernel sizes and concatenate the results, allowing the network to capture features at multiple spatial scales simultaneously. Label smoothing regularization replaced hard 0 and 1 class labels with smoothed versions, preventing the network from becoming overconfident on training examples.
The method was intentionally designed as a binary tumor versus non-tumor classification rather than a four-class problem (tumor positive, tumor negative, non-tumor positive, non-tumor negative). This simplification made the task tractable for the available data. The authors note that the binary output can subsequently be subdivided using standard color deconvolution methods to separate Ki67-positive and Ki67-negative nuclei within the identified tumor regions.
The proposed deep learning method can directly identify tumor boundaries on Ki67 stained slides, reducing the need for pathologists to manually translate tumor regions from H&E slides to Ki67 slides. This step currently adds time and introduces potential for misregistration errors in clinical practice.
By replacing quantum dot double staining and pancytokeratin co-staining methods, the approach dramatically reduces the cost of Ki67 tumor identification. Quantum dot staining is significantly more expensive than standard Ki67 staining and is not widely available, meaning that hospitals currently relying on manual estimation methods could adopt automated tumor identification without any additional reagent costs.
The method also has the potential to reduce pathologist workload by automating the labor-intensive task of identifying tumor hotspots across whole-slide images. Once tumor regions are automatically delineated, standard automated image analysis software can count Ki67-positive versus Ki67-negative cells within those regions to compute the Ki67 index, completing the grading workflow with minimal manual intervention.
Limitations include the relatively small dataset of 33 whole-slide images from a single institution, the focus on well-differentiated NETs with predictable morphology, and the class imbalance between tumor and non-tumor tiles. Future work should validate the method on larger multicenter datasets and on poorly differentiated neuroendocrine carcinomas where the morphological distinction between tumor and non-tumor cells may be less clear.