18F-FDG PET/CT is increasingly used for staging and treatment monitoring across many cancer types. Because whole-body scans routinely cover the chest from mid-orbit to proximal thigh, they frequently image breast tissue even when the primary indication is another cancer or condition. This creates opportunities for incidental detection of breast cancer - one study found 440 incidental breast lesions in 1,951 patients undergoing PET/CT.
The standard way nuclear medicine physicians and radiologists review whole-body PET scans is via maximum-intensity projection (MIP) images - 2-dimensional images created by projecting only the pixels with the highest radiotracer uptake from a 3-dimensional volume onto a flat image. MIP images enable rapid whole-body review but can miss subtle findings or confuse normal physiological uptake with pathological lesions.
Deep learning (DL) algorithms, particularly convolutional neural networks (CNNs), have demonstrated strong performance across many medical imaging tasks including mammography interpretation and ultrasound classification. However, their application to PET/CT MIP images for breast cancer detection had not been well studied before this work.
This study tested whether a DL model trained on PET MIP images from multiple viewing angles could classify whole-body PET/CT scans as positive or negative for breast cancer, and whether using four different projection angles (0, 30, 60, and 90 degrees) improved performance over a single-angle model.
The study retrospectively enrolled 500 female patients from Tokyo Medical and Dental University: 250 with pathologically confirmed breast cancer and 250 without breast cancer or abnormal chest uptake. The breast cancer group predominantly had T1c (1-2 cm) tumors, representing early-stage disease. The dataset was divided into 400 training/validation cases (200 BC, 200 non-BC) and 100 independent test cases (50 BC, 50 non-BC).
For each patient, PET MIP images were generated at four different viewing angles: 0, 30, 60, and 90 degrees. The front-facing (0-degree) view is the standard clinical view. Adding 30, 60, and 90-degree rotations provides additional perspectives that can disambiguate lesions overlapping with high-uptake organs (heart, liver, kidneys) at one viewing angle but not at others.
Images were preprocessed by cropping the top and bottom (removing brain and bladder regions) to focus the model on breast tissue and relevant anatomy. For training data, aggressive data augmentation was applied - randomly varying zoom, rotation, brightness, contrast, and horizontal flip - to create 16 different image variants from each original, yielding 5,120 total training image sets and helping the model generalize better to real-world variation.
Two CNN models were built using the Xception architecture (a 36-layer convolutional network). The 0-degree model used only the standard frontal MIP view. The 4-degree model incorporated all four viewing angles using a creative encoding strategy: the 30 and 60-degree images were combined via pointwise (1x1) convolution, then all four angles were encoded into the red, green, and blue channels of a single RGB image, allowing Xception to process all perspectives simultaneously.
The 4-degree model achieved 96% sensitivity, 80% specificity, and AUC of 0.936 on the independent test set. The 0-degree model achieved 82% sensitivity, 88% specificity, and AUC of 0.918. The five radiologists achieved sensitivities ranging from 80% to 98% and AUCs ranging from 0.872 to 0.967, reflecting the wide variation in performance between less and more experienced readers.
The 4-degree model's AUC was significantly better than the least experienced radiologist (Reader 1: AUC 0.872, p=0.036). Although differences were not statistically significant (possibly due to sample size), the model outperformed Reader 2 (AUC 0.891) and Reader 3 (AUC 0.900). The model was comparable to the two most experienced readers (Reader 4: AUC 0.957 and Reader 5: AUC 0.967).
The 4-degree model correctly diagnosed 6 cases that the 0-degree model missed, including lesions near the nipple and non-mass-like FDG uptake patterns. This demonstrates that the additional viewing angles provide meaningful complementary information - lesions overlapping with high-uptake anatomy from one angle can become clearly visible from another.
The 4-degree model produced 10 false positives and 3 false negatives. False positives were primarily caused by normal physiological FDG uptake in nipples and mammary glands being misread as lesions. False negatives occurred when lesions had very low FDG uptake (SUVmax 0.9-1.2) or when high physiological uptake in adjacent organs masked the cancer signal. Two false-negative cases were ductal carcinoma in situ (DCIS), which typically has low metabolic activity.
The core technical innovation of this study is the method for combining four MIP images into a single input for a CNN designed for 3-channel (RGB) images. Rather than training four separate models or sequentially processing images, the authors used pointwise (1x1) convolution to combine the 30 and 60-degree images into a single composite image. This image was then used as the green channel of an RGB image, with the 0-degree image as red and the 90-degree image as blue.
This approach allows all four angular perspectives to be processed simultaneously within a single forward pass through the Xception network. It requires no architectural modifications to the base network, keeps computational cost manageable, and leverages the learned feature representations from ImageNet pretraining (transfer learning) - where Xception was first trained on millions of general images before being fine-tuned for the specific task of BC classification.
Models were trained using five-fold cross-validation on the training/validation set, with stochastic gradient descent optimization, learning rate decay scheduling, and 30 training epochs per fold. The test set (100 patients) was kept completely independent and never used during training or hyperparameter tuning, ensuring unbiased performance estimates.
The complementary benefit of multiple angles aligns with findings from other imaging domains: Nobashi et al. demonstrated that CNNs using ensemble images from multiple axes improved performance over single-image models for brain PET scans, and ensemble methods generally outperform single-input models in breast image classification tasks. The multi-angle approach essentially implements a form of view ensemble within a single unified model.
The clinical significance of this work is in reducing missed incidental breast cancer findings on whole-body PET/CT scans ordered for other indications. Less experienced radiologists and nuclear medicine physicians reviewing PET scans primarily for staging other cancers may not thoroughly scrutinize breast tissue, particularly for non-mass-like uptake patterns or subtle lesions near physiologically active structures.
AI tools like mammography screening systems (e.g., MammoScreen) have already demonstrated the ability to reduce radiologist workload by up to 70% in some settings. A similar tool for PET/CT MIP review could flag suspicious breast findings for dedicated follow-up, particularly valuable for patients whose primary scan indication might otherwise direct clinical attention away from incidental breast findings.
The false-positive pattern (nipple and mammary gland uptake mistaken for lesions) suggests that future model improvements could include explicit anatomical localization to distinguish normal physiological uptake sites from pathological ones. The false-negative pattern (low-uptake cancers, particularly DCIS) reflects a fundamental physical limitation of FDG PET rather than a model deficiency - very low-grade or non-invasive cancers may simply not accumulate enough FDG to be distinguishable from background.
Interobserver agreement analysis showed that the CNN models and radiologists made similar overall diagnoses (significant positive correlation), but the lower agreement between models and radiologists compared to radiologist-radiologist agreement suggests they may use different decision criteria. Understanding what image features drive model predictions through explainability tools (such as gradient activation maps) could help align machine and human decision-making and build clinical trust.
This study demonstrates that a deep learning model using PET MIP images from four viewing angles can classify whole-body PET/CT scans as positive or negative for breast cancer with diagnostic performance comparable to experienced radiologists (AUC 0.936). The 4-degree multi-angle model outperformed both the single-angle model and less experienced readers, and was statistically non-inferior to expert radiologists.
The study is the first to directly compare CNN-based breast cancer classification from PET MIP images against human radiologists' interpretations, filling an important gap in the DL-for-PET literature. The finding that adding multiple viewing angles improves performance over a single view has practical implications for how PET image AI tools should be designed.
Key limitations include the single-center retrospective design with a relatively small test set (100 patients), the absence of benign lesions (fibroadenomas, papillomas) as confounders in the non-BC group, and potential image quality variation across three different PET/CT scanner models. The performance of the model on ambiguous or borderline cases warrants further study.
Future work should include large-scale multicenter prospective validation, integration of the actual cross-sectional PET/CT slices (not just MIP images) for more detailed analysis, and testing on more diverse populations including patients with benign breast conditions. Clinical implementation could take the form of an automated screening tool that pre-flags MIP images with high probability of breast pathology for dedicated radiologist review, potentially improving both efficiency and diagnostic accuracy.