Lung cancer remains the leading cause of cancer-related deaths worldwide, making early detection of pulmonary nodules a critical clinical priority. Many nodules are discovered incidentally during routine chest imaging, and while most are benign, a subset may represent early-stage malignancies that require prompt intervention.
Low-dose computed tomography (LDCT) screening has demonstrated that it can reduce lung cancer mortality by 20% over seven years in high-risk individuals compared to traditional chest X-rays, leading to large-scale screening programs in the United States. However, this success has created a new challenge: an enormous volume of CT scans that radiologists must interpret.
Computer-Aided Detection (CAD) systems have been developed over the past two decades to help radiologists identify pulmonary nodules faster and more accurately. Yet these systems face major obstacles including the small size of nodules, their variable shapes, low contrast against surrounding tissue, and a severe imbalance between nodule pixels and background pixels in each scan.
This study proposes a federated learning framework that allows multiple hospitals or clinical sites to jointly train an AI model for lung nodule segmentation without ever sharing their patients' raw CT data. Only model parameter updates travel between institutions and a central server, keeping sensitive medical images strictly local.
Federated learning addresses a fundamental barrier in medical AI development: privacy regulations such as HIPAA in the US and GDPR in Europe prevent the pooling of patient data across institutions. Without such pooling, AI models trained at a single site often perform poorly on data from other hospitals that use different scanners or imaging protocols.
The framework combines a novel hybrid neural network architecture with a specially designed loss function to tackle the dual problems of data privacy and segmentation accuracy simultaneously, making it one of the first unified systems to address all three challenges - privacy, class imbalance, and heterogeneous data distributions - in one solution.
The core AI model is a hybrid architecture that merges two powerful approaches: a U-Net encoder-decoder for capturing fine local textures, and Transformer-based self-attention modules for capturing long-range spatial relationships across the entire image. This combination allows the model to understand both the immediate appearance of a nodule and its broader anatomical context.
The encoder pathway uses residual convolutional blocks stacked in three layers to progressively extract increasingly abstract features from the CT image. Skip connections carry spatial detail from the encoder to the decoder, preventing the loss of fine-grained information needed for precise boundary delineation.
The Transformer bottleneck at the deepest part of the network applies multi-head self-attention, allowing every part of the image to attend to every other part simultaneously. This global context is especially valuable for identifying small or subtle nodules that cannot be reliably spotted based on local appearance alone.
The decoder reconstructs a full-resolution segmentation mask using transpose convolutions combined with residual blocks and encoder skip connections. Layer Normalization and Dropout throughout the network stabilize training and improve generalization across the diverse datasets contributed by different federated clients. The entire model contains only 7-8 million parameters, making it lightweight enough for real-world deployment.
Before training, each CT slice undergoes a two-stage preprocessing pipeline to improve image quality. First, Contrast Limited Adaptive Histogram Equalization (CLAHE) is applied with a clip limit of 2.0 and an 8x8 tile grid, which enhances local contrast and makes subtle nodular structures more visible without amplifying background noise.
Following contrast enhancement, pixel intensities are normalized to a standard range using min-max scaling. This ensures that data from different scanners and hospitals, which may have very different raw intensity distributions, are brought to a consistent scale before being fed into the model - a critical step for stable training across federated clients.
The custom loss function combines Dice Loss and Focal Loss in a weighted sum (90% Dice, 10% Focal). Dice Loss directly optimizes the overlap between predicted and ground-truth nodule regions, making it well-suited for the small target areas typical of lung nodules. Focal Loss adds extra weight to difficult, hard-to-classify pixels that simple cross-entropy would overlook.
The Focal Loss parameters (alpha=0.25, gamma=2.0) were chosen empirically to balance the severe class imbalance in lung CT scans, where nodule pixels represent only a tiny fraction of the total image. Together, the combined loss function leads to more stable training, better convergence, and significantly improved segmentation of nodule boundaries compared to using either loss alone.
The federated training process involves five simulated clinical clients, each holding an equal and non-overlapping partition of the LUNA16 dataset. Training occurs over four communication rounds, with each round consisting of 500 local training epochs per client before model weights are sent to the central server.
The Flower framework - an open-source federated learning platform - manages the distributed training process. The server initializes the global model, distributes it to clients, receives locally trained weight updates, and aggregates them using the Federated Averaging (FedAvg) algorithm, which computes a weighted average of all client models.
The LUNA16 dataset was chosen as the benchmark because it originates from multiple hospitals with different scanners, imaging protocols, and annotation styles - naturally mimicking the heterogeneous, non-identically distributed data conditions found in real federated healthcare settings. This makes LUNA16 ideal for evaluating whether federated models can generalize across institutions.
The federated model achieved strong overall performance, with Dice coefficients ranging from 0.74 to 0.93 across the five clients for solid pulmonary nodules in the 15-25 mm size range. Client 2 performed best, achieving a Dice of 0.93, precision of 0.90, recall of 0.95, and an intersection-over-union (IoU) score of 0.86.
Clients 1, 4, and 5 all achieved Dice scores above 0.85, demonstrating that the model can generalize well across sites with representative nodule distributions. Background (non-nodule) regions were segmented with near-perfect accuracy across all clients, with Dice and IoU consistently above 0.999, reflecting the dominant class.
Client 3 was a notable exception, achieving a Dice of only 0.74 and a recall of just 0.59 despite perfect precision (1.00). This pattern - where every predicted nodule pixel was correct but many real nodule pixels were missed - indicates under-segmentation, likely caused by low-contrast or part-solid nodules that are inherently harder to delineate.
FedAvg's weighted aggregation mechanism helped compensate for Client 3's poor performance by giving more influence to the four high-performing clients during global model updates. This emergent robustness shows that federated learning can maintain strong overall performance even when individual participants have challenging data distributions.
Head-to-head comparisons with five baseline architectures trained under identical federated conditions revealed the clear superiority of the proposed hybrid model. The earliest baseline, FCN-8s, achieved a Dice of only 0.31, while PSPNet reached 0.58, 2D Residual U-Net 0.72, 3D U-Net 0.68, and MSS U-Net 0.67.
The proposed Hybrid Transformer-U-Net achieved a Dice of 0.80, approximately 8.5% higher than the strongest baseline. Critically, it also achieved the lowest error rates of all evaluated architectures, with a false positive rate of 0.10 and false negative rate of 0.24, meaning it both minimized false alarms and caught the most real nodules.
The performance gap over 3D U-Net is particularly instructive: despite 3D convolutions having theoretical advantages for volumetric CT data, the 3D model underperformed in federated settings because its large parameter count (30-35 million) creates high communication costs and its volumetric patterns conflict across clients with different slice thicknesses.
The proposed model's lightweight design (7-8 million parameters) compared favorably to both a GAN-based competitor (32-38 million parameters) and 3D Res U-Net (30-35 million parameters), all achieving similar Dice scores around 0.80 - demonstrating that the hybrid model achieves equivalent accuracy with a fraction of the computational cost.
A systematic ablation study tested what happens when individual components are removed or simplified, confirming the contribution of each architectural choice. The full Hybrid Transformer-U-Net served as the reference with a mean Dice of 0.810 +/- 0.030.
Removing the Transformer bottleneck entirely - leaving only convolutional layers - dropped the Dice to 0.685 (p less than 0.001), showing that local convolutions alone cannot capture the global contextual dependencies needed to reconcile variable nodule appearances across different federated clients.
Reducing encoder-decoder depth progressively degraded performance: a two-block architecture scored 0.760 and a single-block architecture scored 0.695. Shallower models lack the representational capacity to capture subtle intensity differences and produce noisier gradient updates that destabilize federated averaging.
A pure Transformer architecture (Swin-UNet) achieved the highest Dice of 0.820, but at the cost of 55 million parameters - seven times larger than the proposed model. Its communication and memory overhead makes it impractical for real-world federated deployment across institutions with varying hardware resources, confirming that the hybrid model offers the best balance of accuracy and deployability.
Four federated optimization strategies were compared under identical experimental conditions: FedAvg, FedMA, FedProx++, and FedOpt (FedAdam). While FedMA achieved the highest Dice score of 0.820, it requires strict architectural homogeneity and neuron-level matching that is computationally prohibitive for Transformer-based models with complex attention heads.
FedAvg was selected as the primary optimizer despite its modest 0.800 Dice score, because it aggregates client updates through simple weighted averaging without architectural assumptions, neuron matching, or extra parameter transfers. This simplicity makes it natively compatible with the hybrid model's mix of attention layers and convolutional blocks.
FedAvg demonstrated an important emergent property: high-performing clients naturally compensate for problematic clients during aggregation. In this study, four reliable clients stabilized the global model despite Client 3's substantial recall deficit, illustrating that FedAvg's simplicity can actually be an advantage in realistic clinical settings where institutions differ greatly.
This work establishes a practical pathway for privacy-preserving AI in lung cancer screening by demonstrating that a federated model can achieve competitive segmentation performance - Dice up to 0.93 - without centralizing sensitive patient data. The framework complies with HIPAA and GDPR by design, since raw CT scans never leave the institution.
The model's performance on solid nodules in the 15-25 mm range is strong, but reduced accuracy on smaller nodules, part-solid nodules, and ground-glass opacities indicates where future work is needed. Advanced preprocessing strategies such as edge sharpening, multi-window CT channel fusion, or multi-scale attention could improve detection of these more challenging nodule types.
The lightweight 7-8 million parameter design makes this framework suitable for deployment not just at well-resourced academic medical centers, but also at community hospitals and edge computing environments with limited bandwidth and hardware - a critical consideration for achieving equitable access to AI-assisted cancer screening.
By combining privacy preservation, rapid convergence, and robust generalization, this federated framework demonstrates that collaborative multi-institutional AI development is feasible without compromising patient confidentiality, pointing toward a future where AI models can continuously improve as more hospitals contribute their locally trained updates.