High-dose-rate (HDR) brachytherapy is a standard treatment for localized prostate cancer in which thin tubes called catheters are inserted directly into the prostate gland. A radioactive source then travels through these catheters to deliver a highly concentrated radiation dose from the inside out.
Between 10 and 18 catheters are implanted into the prostate under ultrasound guidance, and a computer then determines how long the radioactive source should dwell at each position along each catheter to maximize dose to the tumor while protecting surrounding structures like the bladder, rectum, and urethra.
The positioning of catheters is critical because it directly determines what dose distributions are achievable. If catheters are placed suboptimally -- too close to sensitive organs or missing parts of the tumor -- no amount of dwell-time optimization can fully compensate.
Currently, catheter placement relies entirely on the physician's judgment and experience, adapted in real time during surgery while the patient is under anesthesia. This creates significant variability between operators and makes it impossible to predict treatment quality before the catheters are actually inserted.
The operating room is a high-pressure environment where physicians must make rapid decisions. Catheter placement during HDR brachytherapy is often rushed due to time constraints and the presence of an anesthetized patient, increasing the risk of suboptimal needle positioning.
Standard catheter patterns are designed around typical prostate anatomy, but prostate size, shape, and proximity to sensitive organs vary enormously between patients. A one-size-fits-all approach will produce inconsistent results across the patient population.
Knowing the planned catheter positions ahead of time would allow physicians to preview the expected dose distribution before any needles are inserted. This capability -- currently unavailable -- would help guide intraoperative decisions and serve as a quality control benchmark.
Prior research on HDR brachytherapy optimization has focused on refining dwell times after catheters are already placed. Optimal catheter placement prediction combined with dwell-time optimization is a largely unexplored strategy that could substantially improve treatment standardization.
The researchers proposed a three-step framework: (1) deformable image registration using a deep learning registration network (Reg-Net), (2) multi-atlas ranking to identify the most relevant historical patient data, and (3) catheter regression to convert predicted positions into physically straight needle paths.
A key innovation was converting organ shapes into distance maps -- mathematical representations where each point in 3D space is encoded with its distance to the nearest organ surface. This encoding makes it possible for the neural network to understand the spatial relationships between the prostate and surrounding organs even without direct image intensity information.
When a new patient arrives, the system searches an atlas library of past patients and uses Reg-Net to deformably warp each atlas patient's anatomy onto the new patient. It then selects the best-matching atlases using three criteria: prostate volume similarity, multi-organ shape similarity, and catheter position rules (catheters should stay within the prostate and away from the urethra).
The Reg-Net architecture incorporates attention gates -- a mechanism that focuses the network on the most anatomically relevant regions during registration, improving accuracy in critical areas like the prostate-urethra boundary where small errors in catheter prediction would have the largest clinical impact.
After deformable registration transfers catheter positions from atlas patients to the new patient, these positions may be physically unrealistic -- bent at angles that no actual needle can achieve. Catheter regression post-processes these positions to fit straight lines through 3D space, using a geometric tracing algorithm that follows each catheter slice by slice.
The system was trained and evaluated on 90 prostate cancer patients who had previously received HDR brachytherapy. Input to the model was not the CT scan itself but only the organ outlines (binary contour maps) of the prostate, bladder, rectum, and urethra -- making the method adaptable to contours derived from different imaging modalities.
A five-fold cross-validation design was used: patients were split into five groups, and each group was used as a test set exactly once while the other four formed the atlas library. This ensures every patient was evaluated on catheter predictions made without any exposure to their own data.
Plan quality was compared by optimizing radiation source dwell times on both the predicted and the original clinical catheter patterns using identical settings. Both plans were normalized to the same prostate dose coverage (V100 = 95%) to enable a fair comparison of organ-at-risk doses and hotspot metrics.
Before deep learning registration, selected atlas patients matched new patients with Dice similarity scores of 0.83 for the prostate, 0.43 for the urethra, 0.65 for the bladder, and 0.53 for the rectum. After Reg-Net deformable registration, these values improved substantially to 0.95, 0.86, 0.93, and 0.86 respectively -- demonstrating that the network reliably aligns atlas anatomy to new patients.
Most importantly, plans based on the predicted catheter patterns met all standard clinical dose constraints on average when prostate coverage was set to V100 = 95%. This means the predicted catheter placements were clinically acceptable and not just geometrically similar to the physician-placed catheters.
The predicted plans showed slightly higher dose hotspots within the prostate: V150 was higher by 5.0% and V200 by 2.9% on average compared to the clinical plans. These metrics reflect areas where the dose exceeds 150% and 200% of the prescribed level -- both remained within acceptable clinical ranges.
For all three critical organs-at-risk -- bladder, rectum, and urethra -- the average dose metric differences between predicted and clinical plans were close to zero, and the range for most patients fell within a clinically negligible margin of plus or minus 1 cubic centimeter.
The model's accuracy depends on the diversity of the atlas library. If a new patient's anatomy is substantially different from all cases in the library -- for example, an unusually large prostate or an uncommon proximity to the rectum -- the predicted catheter pattern may be less accurate. Expanding the atlas database with more diverse patient data is an important next step.
The registration network is trained without ground-truth deformation data, making it inherently an ill-posed optimization problem. The model must regularize its solution using mathematical constraints, but guaranteeing physically realistic deformations without reference data remains challenging. Adding biomechanical constraints to the model could improve realism.
Currently, the model uses only contour maps of the prostate and organs-at-risk as input. Including actual CT or MRI image data could improve the accuracy of both the registration step and the atlas ranking step by incorporating tissue contrast information not captured in organ boundaries alone.
Although the model predicts catheter positions using pre-procedure CT contours, the authors note that using intraoperative TRUS-derived contours would be more accurate because the patient's anatomy is in the same position as during catheter insertion. Combined with automatic organ segmentation tools, this could enable fully real-time intraoperative guidance.
This study demonstrates for the first time that deep learning can predict clinically acceptable catheter placement patterns for HDR prostate brachytherapy before any needles are inserted. The system can generate a predicted catheter plan in approximately 5 minutes after organ contours are provided.
By providing physicians with a proposed catheter map prior to surgery, the method could serve as a quality assurance tool -- helping operators anticipate achievable dose distributions, identify potential placement challenges, and reduce decision-making variability across different surgeons and institutions.
The approach reduces -- but does not eliminate -- dependence on physician experience. Skilled operators will still interpret and adapt the prediction based on real-time anatomy, but they gain an evidence-based starting point derived from the cumulative experience embedded in the atlas library rather than relying solely on individual heuristic judgment.
Future extensions could incorporate dominant intraprostatic lesion mapping from MRI to enable focal dose boost planning, where radiation is escalated specifically to the most aggressive areas within the prostate -- a precision medicine approach increasingly supported by clinical evidence.