Positron emission tomography (PET) is a nuclear imaging technique that maps metabolic activity in the body. Cancer cells typically consume far more glucose than normal cells, so injecting a radioactive glucose-like tracer (FDG) and scanning with PET can reveal tumor locations - including in the lungs, where PET is commonly used for staging and treatment planning.
Accurate tumor delineation in PET is critical for radiation therapy planning and for measuring tumor characteristics called radiomic features. However, PET images inherently have lower spatial resolution and higher noise than CT or MRI scans, making it difficult to precisely define where a tumor ends and healthy tissue begins.
Manual segmentation - having a radiologist draw tumor boundaries slice by slice - is the current standard but is tedious, expensive, and highly variable between different readers. Studies show significant inconsistency in how different doctors delineate the same tumor, which can affect treatment decisions.
Partial volume effects (PVEs) are a specific problem in PET where the blurring from the scanner's limited resolution causes tumor boundaries to appear smeared, making tumors appear larger or smaller than they are. Existing automated methods struggle with PVEs, especially for small tumors.
Deep learning approaches have shown great promise for segmenting tumors in CT and MRI images, but applying them to PET faces additional obstacles: PET datasets are much smaller (fewer patients are scanned by PET than CT), and manually drawn tumor boundaries used as training targets carry the same inaccuracies and variability as manual clinical practice.
To address these combined challenges, the researchers developed a three-module deep learning framework specifically designed for PET tumor segmentation. Rather than relying solely on limited, imperfect clinical training data, the framework cleverly uses physics-based image simulation to generate thousands of realistic synthetic training images with perfectly known tumor boundaries.
Module 1 generates large numbers of realistic simulated PET images containing synthetic tumors placed in actual patient background lung images. The simulated tumors are created using statistical models of real tumor properties - size, shape, intensity, and internal heterogeneity - learned from clinical data, so they closely match what real lung tumors look like in PET scans.
Module 2 trains a deep learning network called a modified U-net (mU-net) on the simulated images where ground-truth tumor boundaries are perfectly known. This allows the network to learn the fundamentals of PET tumor segmentation from abundant, accurate examples rather than from a small set of imprecise manual annotations.
Module 3 fine-tunes the pre-trained network using a small set of real patient images with radiologist-defined boundaries as approximate ground truth. This step allows the network to learn subtle features of real tumors that may not be fully captured in simulations, bridging the gap between synthetic and clinical data.
The simulation approach begins by extracting statistical properties of real tumors from 160 lung cancer patients scanned on FDG-PET. These properties include tumor size, shape, internal intensity patterns, the ratio of tumor brightness to surrounding tissue, and the degree of internal heterogeneity (non-uniform uptake that can indicate areas of tumor necrosis or variable metabolism).
Kernel density estimation (KDE) was used to model the statistical distribution of each tumor property, capturing the full range of variation seen in real patients rather than just average values. New synthetic tumors are then generated by randomly sampling from these learned distributions, ensuring the simulated dataset covers a realistic range of tumor appearances.
Synthetic tumors are composited onto real PET background images from patients who did not have tumors visible in those lung regions. This ensures the background tissue texture is fully realistic. The placement is followed by a physics simulation of the PET scanner itself - including its blurring effect (modeled as a 5mm Gaussian) and realistic noise levels - to ensure the final simulated images look indistinguishable from real clinical scans.
A board-certified radiologist visually confirmed the realism of the generated images, and 10,000 simulated image slices were generated from 113 patient backgrounds to serve as the primary training dataset. This approach avoids the key limitation of conventional data augmentation, which still relies on imperfect manual boundaries as ground truth.
The core network is a modified U-net (mU-net), a deep learning architecture originally designed for biomedical image segmentation. U-net uses an encoder-decoder structure: the encoder compresses the image into abstract feature representations, and the decoder expands those features back to produce a pixel-by-pixel segmentation map showing which pixels belong to the tumor versus background.
Skip connections link corresponding layers in the encoder and decoder, allowing the network to combine fine-grained spatial details (from early layers) with higher-level contextual understanding (from deeper layers). In this implementation, skip connections use element-wise addition rather than the concatenation used in the original U-net, which the authors found to stabilize training and improve performance.
Key modifications from standard U-net include dropout regularization (randomly disabling neurons during training to prevent the network from memorizing training examples), a leaky rectified linear unit (ReLU) activation function to prevent gradient vanishing, and a class-weighted loss function that penalizes errors on tumor pixels more heavily than background pixels - important because tumor regions are much smaller than background in most image slices.
Hyperparameter optimization was conducted through five-fold cross-validation on simulated data, with a grid search across several orders of magnitude for each parameter. After selecting optimal settings, the network was retrained from scratch on the full simulated dataset before proceeding to the clinical fine-tuning step.
The clinical dataset consisted of FDG-PET scans from 160 patients with biopsy-proven lung cancer at a mean age of 63.2 years. Images were acquired across five different PET scanner models (Discovery LS, RX, HR, ST, and STE), providing an opportunity to test whether the framework generalizes across different equipment - a critical practical requirement for real-world clinical adoption.
Manual segmentations were performed slice-by-slice by a radiologist with experience in nuclear medicine and diagnostic radiology. These served as surrogate ground truth for training the fine-tuning step and for evaluating performance on patient images. The primary performance metric was the Dice Similarity Coefficient (DSC), where a value of 0.7 or above is generally considered high segmentation accuracy.
Multiple evaluation experiments were designed to test different aspects of the framework: accuracy versus established semi-automated methods, generalizability across scanners, performance with different numbers of training images, and sensitivity to partial volume effects. Each experiment used completely independent training and test sets.
Comparison methods included commonly used thresholding approaches (segmenting tumors based on fixed percentages of maximum tumor signal - 30%, 40%, and 50%), the active contour snake method, and a probabilistic Markov random fields-Gaussian mixture model approach. All competing methods required manual input (such as a seed pixel or region of interest) while the proposed framework was fully automated.
On simulated images with known ground truth, the framework achieved a mean Dice Similarity Coefficient (DSC) of 0.87, significantly outperforming all competing semi-automated methods, which ranged from 0.57 to 0.63 (p less than 0.001). When only cases where the tumor was correctly located are considered, DSC improved to 0.91, indicating near-perfect boundary delineation once the tumor is found.
On real patient images, the framework achieved a mean DSC of 0.73, again outperforming all comparators (best competing method: 0.68 by MRF-GMM). The framework successfully segmented the smallest tumor cross-section of 1.83 cm squared and handled challenging cases including tumors near the heart, tumors with irregular shapes, and tumors surrounded by other high-uptake structures.
Generalizability testing showed that when trained on images from one scanner type and tested on five others, the framework maintained a DSC of 0.74 - essentially the same performance as when training and testing occurred on the same scanner type. Tested in the reverse direction (trained on multiple scanners, tested on one), DSC was 0.71. This scanner-agnostic performance is a major practical advantage.
The framework's tumor localization accuracy - the fraction of cases where it correctly identified a tumor's general location - reached over 80% in most experiments and up to 91% when using the largest training set of 104 patients, even though this was performed automatically without any user-provided tumor location or seed point.
A key advantage of the simulation-based approach is dramatically reduced dependence on large clinical training sets. The framework achieved a DSC of 0.70 when fine-tuned on just 30 patient images - a result that is clinically meaningful and comparable to the performance of methods that require hundreds or thousands of training cases.
Training with as few as one patient image still resulted in a tumor localization accuracy of 72-74%, demonstrating that the physics-guided pre-training from Module 1 gives the network a strong foundation that requires very little clinical adaptation. A conventional mU-net trained exclusively on clinical data consistently underperformed across all training set sizes.
Partial volume effect (PVE) analysis on 2,000 simulated images showed that the framework achieved a mean DSC of 0.91, compared to 0.75 for PVE-affected boundaries (the blurred tumor outlines produced by the scanner's limited resolution). The framework was substantially more accurate at recovering true tumor boundaries despite blurring.
Tumor area estimation - the ratio of predicted to true tumor size - was also more accurate with the proposed framework across all tumor sizes, including small tumors where PVEs are most severe. This matters clinically because accurate tumor volume measurement is used to assess treatment response and plan radiation doses.
The physics-guided simulation approach offers a fundamental advantage over other data augmentation strategies. Simple augmentations like rotation or scaling still rely on imperfect manual boundaries as ground truth, while generative adversarial networks (GANs) can be unstable and don't incorporate PET physics. The KDE-based physics simulation avoids both problems while generating unlimited perfectly labeled training data.
Fully automated operation distinguishes this framework from all competing methods tested, which required manual input such as a seed point or region of interest. Automation removes inter-user variability in the segmentation workflow and makes the approach practical for routine clinical deployment or large-scale population studies.
The three-module design is modular and extensible - the simulation module could be adapted for other low-resolution modalities like SPECT or optical imaging, and the fine-tuning module could potentially incorporate feedback from radiologists in a reinforcement learning loop for cases where the automated result is uncertain.
Combining PET with CT or MRI data is a natural extension the authors did not pursue here since the framework was designed for PET-only scenarios, but they acknowledge that multimodal approaches and time-of-flight PET systems (which provide sharper images) could further improve performance when integrated with the proposed modular structure.
Current limitations include the framework's 2D slice-by-slice operation rather than full 3D volume segmentation. While the framework can process all tumor-containing slices to reconstruct a 3D tumor volume, a native 3D approach could exploit volumetric contextual information to improve performance, particularly in cases where 2D localization fails.
Tumor location modeling in the simulations is partially manual - radiologists pre-selected possible tumor locations in background images. This means the simulation doesn't fully capture the biological patterns of where tumors are most likely to occur, which could improve localization accuracy if incorporated. Additionally, the framework currently segments only one tumor per image slice.
The study was conducted at a single institution, using data from one center even though multiple scanner types were included. Multi-institutional validation would strengthen confidence in the generalizability claims, and the authors suggest future evaluation across different institutions with varying imaging protocols.
The open-source code for the framework and supplementary data have been made publicly available, facilitating further research, validation in new datasets, and potential integration into clinical workflows. Future work could extend the approach to other cancer types where PET segmentation is clinically important, such as lymphoma and head-and-neck cancers.