3D Unsupervised deep learning method for magnetic resonance imaging-to-computed tomography synthesis in prostate radiotherapy

Phys Imaging Radiat Oncol 2024 Deep Learning 7 Explanations View Original
Original Paper (PDF)

Unable to display PDF. Download it here or view on PMC.

Plain-English Explanations
Page 1
The MRI-Only Radiotherapy Problem

Radiation therapy for prostate cancer requires an extremely precise map of the patient's anatomy. Historically, computed tomography (CT) has been the standard imaging modality for treatment planning because it directly measures electron density -- the property that determines how radiation is absorbed by different tissues. Without accurate electron density information, dose calculation is impossible.

However, magnetic resonance imaging (MRI) offers substantially better soft tissue contrast than CT, making it more accurate for defining where the prostate begins and ends, and where critical structures like the rectum and neurovascular bundles are located. This has led to the clinical adoption of MR-Linac devices that integrate MRI with a linear accelerator for daily guided treatment, and growing interest in MRI-only radiotherapy workflows where no CT is acquired at all.

The fundamental obstacle to MRI-only radiotherapy is that MRI does not provide the electron density information needed for dose calculation. The proposed solution is to generate a synthetic CT (sCT) from MRI images using deep learning -- an AI that learns to predict what a CT scan would look like based on the corresponding MRI. If accurate enough, this synthetic CT can replace the actual CT scan for treatment planning purposes.

Eliminating the CT scan from radiotherapy workflows has multiple benefits: patients receive less radiation exposure from planning scans, fewer imaging appointments are required, and MRI-to-treatment workflows become simpler and faster. For patients treated on MR-Linac systems, daily MRI can be used not just for positioning but potentially for daily dose monitoring if accurate synthetic CTs can be generated from routine treatment images.

TL;DR: MRI-only radiotherapy workflows require synthetic CT images generated from MRI to enable dose calculation, potentially eliminating CT scanning and its associated radiation exposure while enabling MRI-guided adaptive treatment.
Pages 1-2
Supervised vs. Unsupervised AI: Why It Matters

Deep learning methods for MRI-to-CT synthesis fall into two categories. Supervised methods train on paired MRI-CT datasets from the same patient, where the AI learns to predict CT values from corresponding MRI voxels. These methods generally perform well but have a critical practical limitation: they require precise registration of MRI and CT images, because the two modalities cannot be acquired simultaneously and patient anatomy shifts between scans.

CT-MRI registration errors propagate directly into training data inaccuracies, degrading model performance in proportion to registration quality. Creating a large paired dataset also requires significant resources -- expert delineation of structures in both modalities, deformable registration computation, and quality control. This makes scaling supervised approaches to multicenter data technically challenging.

Unsupervised methods avoid this problem entirely by training without requiring matched pairs. A common approach uses conditional Generative Adversarial Networks (cGANs), where a generator network learns to produce CT-like images from MRI while a discriminator network evaluates whether the output is realistic. The generator improves by trying to fool the discriminator; the discriminator improves by trying to detect fakes.

The challenge with unsupervised learning in medical imaging is training instability. Without explicit ground truth to compare against, the generator can converge to unrealistic outputs. This study addressed this problem by introducing a novel perceptual loss function -- the CREPs loss -- that guides the unsupervised generator to produce outputs matching the visual style of CT scans without requiring pixel-level alignment with a paired CT.

TL;DR: Supervised CT synthesis requires precise MRI-CT registration that is difficult to achieve in practice; unsupervised methods avoid this requirement but suffer from training instability that a new perceptual loss function was designed to address.
Pages 2-4
Three Centers, Three Architectures, One Framework

The study used data from 99 prostate cancer patients across three institutions in France and Australia, each using different imaging systems. Center 1 (39 patients) used a 3T Siemens MRI and two different CT scanners. Center 2 (30 patients) used a 1.5T MRI. Center 3 (30 patients) used a 0.35T MRI-Linac system -- a substantially lower field strength reflecting real-world clinical diversity. This multi-scanner, multicenter design specifically tested whether AI models could generalize across imaging environments.

Three network architectures were compared: a supervised model trained on registered MRI-CT pairs, an unsupervised unpaired model trained where each iteration used CT images from one patient and MRI from a different patient (no pairs at all), and an unsupervised paired model that used the same patient's MRI and CT but without requiring precise registration alignment. All three used the same 3D conditional GAN backbone based on the Pix2Pix architecture, with a 6-block ResNet generator.

The key innovation was the CREPs loss function, built on a ConvNext-Tiny network. For supervised training, only the content term was used -- ensuring structural anatomical features were preserved. For unsupervised training, only the style term was used -- constraining the generator's output to produce CT-like tissue appearance (correct Hounsfield unit ranges and texture) without requiring anatomical point-to-point correspondence. This separation of content and style guided training without registration requirements.

Models were trained and tested under three scenarios: monocenter (trained and tested on same institution data), unseen center (trained on one center, tested on a different one), and multicenter (trained on all three centers combined). The multicenter model represents the most clinically practical scenario, where a single shared model is deployed across institutions. Dose evaluation used a 60 Gray prostate treatment plan with volumetric modulated arc therapy, comparing dose distributions calculated on actual CT versus synthetic CT.

TL;DR: Ninety-nine patients from three centers with different MRI scanners (0.35T to 3T) were used to compare supervised and two unsupervised synthetic CT generation approaches, evaluated on both image accuracy and actual radiation dose calculation performance.
Pages 4-5
Image Accuracy Across Methods and Structures

All three methods produced visually realistic synthetic CT images of the prostate region. The Mean Absolute Error (MAE) -- the average difference in Hounsfield units between the synthetic and real CT -- ranged from 33.6 to 45.5 HU across the whole body volume. The unsupervised paired model achieved the lowest MAE of 33.6 HU, slightly better than the supervised model (37.4 HU). The unsupervised unpaired model was least accurate at 45.5 HU.

Performance varied by tissue type. Soft tissue structures -- the prostate (27-32 HU MAE), bladder (27-43 HU), and rectum (34-44 HU) -- all showed lower errors than bone, which had MAE values of 100-159 HU. This is expected: MRI provides excellent soft tissue discrimination but does not directly encode bone mineral density, making bone the hardest tissue to synthesize accurately from MRI alone.

Rectum showed higher variability than other structures, primarily due to the presence of gas pockets. Gas appears on MRI but may differ in location or volume between the MRI and CT acquisitions, and no deep learning algorithm currently reconstructs gas reliably. This is an inherent limitation of all MRI-to-CT synthesis methods, not specific to this study's approach.

The multicenter model performed comparably to monocenter models in most scenarios. When a monocenter model was tested on data from a different center (unseen center scenario), accuracy degraded -- confirming that models trained on one institution's data do not automatically generalize. The multicenter model specifically overcame this limitation, achieving similar performance across all three centers despite their different scanner types and field strengths.

TL;DR: The unsupervised paired model achieved the best overall image accuracy (MAE 33.6 HU), soft tissues were synthesized more accurately than bone, and multicenter training enabled consistent performance across diverse scanner environments.
Pages 4-5
Dosimetric Validation: What Matters for Treatment

Image accuracy metrics like MAE do not directly answer the clinical question of whether a synthetic CT is good enough for treatment planning. The key validation is whether radiation dose distributions calculated on the synthetic CT match those calculated on the real CT. If dose errors are small, the synthetic CT can safely replace the real scan.

All three methods achieved clinically acceptable dose accuracy. Gamma pass rates -- a standard metric comparing dose distributions point-by-point -- exceeded 94% for all methods using the strict 1%/1mm criteria. The unsupervised paired model performed best, with gamma pass rates above 98%. These values are comparable to or better than published benchmarks for other sCT generation methods.

Absolute dose differences for organs at risk were small. Mean dose differences in the bladder were below 0.06 Gray and in the rectum below 0.1 Gray -- far below the level of clinical significance in a 60 Gray treatment. The prostate D95% difference (the dose covering 95% of the prostate volume) averaged 0.31 Gray, a less than 0.5% difference from the reference CT-based calculation.

An important finding was that dose accuracy did not correlate linearly with image accuracy. The MAE between different methods varied substantially, yet their dose calculation performance was nearly identical. This decoupling suggests that dosimetric validation -- not just image quality metrics -- is essential for evaluating sCT methods for clinical deployment. A method with slightly higher MAE may still be perfectly adequate for treatment planning purposes.

TL;DR: All methods achieved gamma pass rates above 94% with dose differences well below clinical significance, and dose accuracy did not correlate with image accuracy -- confirming dosimetric validation is the appropriate benchmark for clinical deployment decisions.
Pages 4-5
Eliminating CT from Radiotherapy Workflows

The core practical implication of this study is that unsupervised synthetic CT generation is ready for clinical consideration in prostate radiotherapy. By eliminating the requirement for precise CT-MRI registration -- the primary bottleneck for supervised approaches -- unsupervised methods enable simpler data pipelines and more scalable multicenter deployment. Centers do not need to invest in complex deformable registration infrastructure to participate in shared model training.

The successful use of data from three centers with field strengths ranging from 0.35T to 3T demonstrates that a single multicenter model can handle significant scanner heterogeneity. The preprocessing steps applied -- including N4 bias field correction, histogram matching, and gradient anisotropic diffusion filtering -- were effective at normalizing cross-center MRI variability, allowing the AI to learn anatomy rather than scanner-specific artifacts.

An exciting extension identified by the authors is daily dose monitoring. Because MR-Linac systems acquire MRI images before each treatment fraction, a validated sCT generation model could convert these daily images into synthetic CTs for real-time dose accumulation tracking. This would allow clinicians to monitor the cumulative dose actually delivered to the tumor and surrounding organs throughout the treatment course -- a capability currently not routinely available in clinical practice.

Study limitations include the relatively small cohort (99 patients across three centers) and the absence of unusual anatomical cases such as patients with hip prostheses or prior organ removal. Deep learning models require diverse training data to handle edge cases robustly, and future work should expand to more centers and include these anatomically challenging scenarios before widespread deployment.

TL;DR: Unsupervised multicenter synthetic CT generation is clinically viable for prostate radiotherapy, eliminates complex registration pipelines, generalizes across diverse scanners, and opens pathways for daily MRI-based dose monitoring.
Page 5
Toward Registration-Free MRI-Only Prostate Radiotherapy

This study demonstrates that 3D unsupervised cGAN-based synthetic CT generation using the CREPs perceptual loss achieves dosimetric accuracy comparable to supervised approaches, without the registration requirements that limit supervised methods in clinical practice. All tested architectures produced clinically acceptable sCTs for prostate radiotherapy dose calculation.

The multicenter training strategy proved effective at building robust models that generalize across institutions and scanner types. This is a prerequisite for any deep learning approach intended for broad clinical deployment -- a model that only works on data matching its training distribution is of limited practical value in the diverse real-world healthcare environment.

The ability to generate accurate sCTs without CT acquisition removes a significant barrier to fully MRI-based prostate radiotherapy workflows. Patients benefit from reduced imaging burden and radiation exposure, while institutions can streamline their treatment planning processes. For MR-Linac users in particular, the integration of sCT generation into daily imaging workflows could enable continuous dosimetric adaptation.

Future development should focus on expanding training datasets to include more diverse patient anatomies and scanner types, validating performance in prospective clinical settings, and extending the approach to other treatment sites where MRI-only workflows would be beneficial. The combination of unsupervised training, multicenter learning, and a principled perceptual loss function represents a technically mature foundation for the next generation of MRI-guided radiotherapy planning tools.

TL;DR: Unsupervised 3D synthetic CT generation with the CREPs loss achieves clinical-grade dosimetric accuracy, enables multicenter deployment without registration dependencies, and lays the groundwork for eliminating CT scanning from prostate radiotherapy workflows.
Citation: Open Access, . Available at: PMC11332181.