Prostate cancer treatment often uses brachytherapy -- a procedure where radioactive seeds or catheters are placed directly inside the prostate to deliver targeted radiation. Ultrasound (TRUS) guides these procedures in real time because it is fast, inexpensive, and does not involve radiation. However, TRUS images have poor soft tissue contrast, making it difficult to precisely identify the exact location of a tumor within the prostate.
Multi-parametric MRI (mpMRI) provides much better detail about tumor location and boundaries, but it cannot be used for real-time guidance during procedures. The solution is to fuse the MRI scan (taken before the procedure) with the real-time ultrasound image during treatment -- overlaying the detailed tumor map from MRI onto the live ultrasound view.
The critical challenge is that the prostate changes shape between the time the MRI is taken and when the ultrasound is used during the procedure. The ultrasound probe itself pushes and deforms the prostate, and the bladder and rectum filling level also change. A simple rigid registration (shifting and rotating the images) is insufficient -- a non-rigid (deformable) registration is needed to account for how the prostate has physically changed shape.
Current manual non-rigid registration methods are physician-dependent, time-consuming, and non-reproducible. Automated approaches have struggled because MRI and ultrasound images look so different that standard image-intensity matching methods do not work well across these two modalities.
Rather than trying to match MRI and ultrasound images pixel by pixel (which fails due to their different appearances), this study represents the prostate as a 3D point cloud -- a set of thousands of 3D coordinates that describe the shape of the prostate both on its surface and throughout its interior volume. The prostate was first automatically segmented from both MRI and TRUS scans using separate deep learning networks, then converted into volumetric point clouds using tetrahedron meshing.
Traditional deep learning architectures work on regular grids (images), but point clouds are irregular -- points are not on a fixed grid and can vary in number. The study used a network architecture inspired by PointNet, which is specifically designed to process irregular 3D point data. The network is designed to be invariant to the order and rotation of points, ensuring it correctly interprets the prostate shape regardless of orientation.
The proposed network, called ProRegNet, takes two point clouds as input -- one from the MRI and one from the TRUS -- and predicts the motion vector for every point in the MRI cloud that would move it to match the TRUS cloud. This effectively maps how every part of the prostate must shift to account for the deformation between imaging sessions.
Unlike conventional CNN approaches that extract features from image intensities, ProRegNet extracts local and global geometric features from the 3D spatial relationships among points. This shape-based representation is more robust to the modality differences between MRI and ultrasound than intensity-based methods.
The key innovation in this study is that the network was trained on deformation fields generated by finite element (FE) simulation -- a computational physics method that models how a material deforms under applied forces. The prostate was treated as a soft tissue with known mechanical properties (Young's modulus, Poisson's ratio) in a hyperelastic material model.
For each patient, surface correspondences between the MRI and TRUS prostate shapes were established using a modified iterative closest point (ICP) algorithm. These surface correspondences were then used as boundary conditions in the FE model, which computed how the interior of the prostate deforms in a physically plausible way when the surface changes shape.
By training ProRegNet on FE-generated deformation fields rather than on manually annotated correspondences or purely data-driven examples, the network implicitly learned the biomechanical constraints governing how soft tissue behaves. This means the predicted deformations respect the physical properties of prostate tissue -- they compress and stretch realistically rather than generating impossible folding or tearing.
For 50 patients, 10 FE simulation sub-steps per patient were used to generate 450 training datasets with diverse deformation magnitudes. The entire registration pipeline -- including segmentation, meshing, network prediction, and final image transformation -- takes 3 to 5 minutes per case, compared to much longer manual processes.
ProRegNet was compared against two baseline methods: a surface-based registration method (SurfReg) that matches prostate surfaces using interpolation, and a standard CNN-based method (CnnReg) that predicts deformation from spatially encoded MRI and TRUS prostate masks.
For surface matching metrics (DSC, MSD, HD), SurfReg performed best -- as expected, since it was explicitly designed to match surfaces. However, the clinically most important metric is target registration error (TRE), which measures how accurately specific internal landmarks (cysts, calcifications) are aligned after registration. On TRE, ProRegNet achieved the best accuracy, with a mean of 1.57 mm -- outperforming CnnReg by 0.91 mm and SurfReg by 1.13 mm.
Visualization of individual cases showed that ProRegNet achieved the best matching of internal structures such as cysts, even when its surface matching was slightly inferior to SurfReg. This demonstrates that for clinical applications (such as accurately targeting a tumor during brachytherapy), matching the interior of the prostate is more important than perfectly matching the surface boundary.
Physical fidelity of the predicted deformations was evaluated through Jacobian determinant analysis. All methods produced deformation fields without singularities (negative Jacobian values indicating impossible tissue folding). ProRegNet generated the smoothest, most physically plausible deformations -- its Jacobian error was significantly lower than SurfReg, confirming that the biomechanical training produced realistic tissue behavior.
The primary clinical application is MRI-guided prostate brachytherapy. In high-dose-rate (HDR) brachytherapy, catheters are inserted through the perineum into precise locations within the prostate to deliver concentrated radiation. If MRI-identified tumor locations can be accurately mapped onto the real-time ultrasound view, the radiation can be preferentially boosted to the dominant intraprostatic lesion (DIL) -- the main tumor focus -- rather than irradiating the entire prostate uniformly.
Accurate DIL localization requires registration error below the typical 3-5mm margin used in radiation treatment planning. The achieved mean TRE of 1.57 mm is within this range, suggesting ProRegNet could support clinically meaningful dose escalation to the tumor while protecting surrounding structures like the urethra and rectum.
The framework also has applications in MRI-targeted prostate biopsy, where fusion of MRI and real-time ultrasound guides biopsy needle placement to suspicious lesions identified on MRI. Accurate non-rigid registration is essential because a needle positioned to a surface-matched location may still miss an internal tumor if the prostate interior has deformed differently from its surface.
The 3-5 minute total processing time compares favorably to manual registration workflows and is fast enough for clinical use -- brachytherapy and biopsy procedures typically allow time for such pre-procedure image fusion before the intervention begins.
This study presents ProRegNet, a novel deep learning framework for non-rigid MRI-ultrasound prostate registration that uniquely combines point cloud neural networks with biomechanical physics training to achieve both clinically accurate and physically realistic deformable registration.
The key contributions are: fully automatic prostate segmentation from both modalities, conversion of 3D prostate shapes to volumetric point clouds, finite element simulation as the training data generator, and a PointNet-derived architecture that learns biomechanically constrained deformation patterns.
The framework overcomes two major limitations of previous approaches: standard CNNs cannot effectively learn registration from featureless binary mask images, and interpolation-based surface methods produce biomechanically unrealistic internal deformations. ProRegNet addresses both by operating on 3D geometric features and learning from physics-based ground truth.
Future work will focus on validating the approach on larger patient cohorts, extending to other clinical interventions such as cryotherapy and focal laser ablation, and incorporating real-time adaptation to account for prostate motion during interventions. The framework is also potentially applicable to other soft tissue organs where deformable registration is needed for image-guided procedures.