Intensity Modulated Radiation Therapy (IMRT) is a precise form of radiation treatment that shapes radiation beams to conform to a tumor's contour, sparing surrounding healthy tissue. For prostate cancer, it is one of the most widely used approaches.
Creating a high-quality IMRT plan requires a human specialist to repeatedly adjust a set of treatment planning parameters (TPPs) -- including weights and threshold doses for organs at risk -- using a treatment planning system (TPS). This trial-and-error process is time-consuming and highly dependent on the planner's experience.
Because plans are often finalized under time pressure, a planner may accept a plan that could still be improved. This variability in plan quality creates a strong motivation to develop automated, intelligent planning tools that can consistently produce high-quality plans.
Researchers at UT Southwestern Medical Center developed a Virtual Treatment Planner Network (VTPN) -- a deep neural network trained to mimic the decision-making behavior of an experienced human planner during IMRT treatment planning.
The VTPN operates by observing the current state of a treatment plan (represented as dose volume histograms, or DVHs) and deciding how to adjust the treatment planning parameters to improve plan quality. This loop of observation and adjustment mirrors exactly how a human planner works.
The key innovation is that this process is learned rather than programmed. Using a framework called deep reinforcement learning (DRL) -- the same technology that enabled computers to master complex games like Go -- the VTPN learns a planning policy entirely from experience, without being told explicitly what rules to follow.
The VTPN is built around a Q-learning framework, a type of reinforcement learning in which the network learns to estimate the long-term value of taking a particular action in a given state. The network selects whichever action is predicted to maximize cumulative plan quality improvement over time.
The network consists of five subnetworks -- one for each of the five treatment planning parameters it controls (such as overdose penalties for the bladder and rectum). Each subnetwork can recommend one of five possible adjustments: increase or decrease by 50%, increase or decrease by 10%, or leave unchanged.
A reward function drives the learning process. After each parameter adjustment, the system generates a new plan and measures the improvement using a clinical scoring system called ProKnow, which evaluates how well a plan meets specific dose constraints for the tumor and surrounding organs. Positive rewards reinforce good adjustments; negative rewards discourage poor ones.
Training used a modified version of the ProKnow score to better reflect clinical priorities -- for example, placing extra emphasis on limiting maximum tumor dose and continuously rewarding reductions in organ dose even beyond strict thresholds.
The VTPN was trained on just 10 prostate cancer patient cases over 100 training episodes. Although the number of patient cases is small, the DRL process generates a large volume of training data by producing many state-action-reward samples during iterative planning simulations.
An epsilon-greedy exploration strategy was used during training: the network initially explores randomly (choosing actions at random) and progressively shifts toward exploiting its learned policy as training matures. This prevents the network from prematurely settling on suboptimal strategies.
Training took approximately one week on a desktop workstation equipped with Intel Xeon CPUs and two Nvidia GPU cards. The system is implemented in Python with TensorFlow and uses an in-house optimization engine based on the alternating direction method of multipliers (ADMM).
The trained VTPN was tested on 64 new prostate cancer cases that were not seen during training. Starting from default (unity) parameter settings, plans with an average ProKnow score of only 4.97 were submitted to the VTPN for optimization.
After the VTPN completed its automated planning process, the average ProKnow score improved to 8.44 out of a maximum of 9 -- a dramatic and consistent improvement across all test cases. The standard deviation of just 0.48 indicates the VTPN reliably produces high-quality plans regardless of the individual patient.
As a benchmark, an experienced human physicist achieved an average ProKnow score of approximately 8.5 on the same cases -- essentially identical to the VTPN. The automated process completed planning in roughly 3 minutes per patient, compared to the much longer time typically required by a human planner.
In a representative test case, the VTPN followed a logical, human-like sequence of decisions. It first focused on reducing dose to the rectum by lowering threshold parameters over the first six steps, then shifted attention to reducing bladder dose in subsequent steps.
After organ sparing was addressed, the VTPN turned to adjusting the penalty weights for both organs to fine-tune the balance. Finally, it increased a parameter controlling PTV (tumor target) dose homogeneity, ensuring the tumor itself received a consistent and adequate dose throughout.
The VTPN autonomously halted the optimization process after 26 steps when it determined that no further parameter changes would improve the plan. The ProKnow score in this example rose from 5.34 to 8.70 -- nearly the maximum possible score of 9.
The current VTPN operates as a black box -- it makes effective decisions but offers little insight into why particular parameter adjustments were chosen. Making the underlying decision logic interpretable to clinicians is an important future goal that will be necessary for clinical adoption.
The system was developed and tested for prostate cancer IMRT, which involves a relatively small number of critical organs. Extending it to more complex treatment sites (such as head and neck cancer) with many more organs at risk will require scaling up the VTPN and significantly increasing computational resources.
Future improvements may include integrating 3D dose image information rather than relying solely on DVHs, incorporating physician preferences learned through inverse reinforcement learning, and coupling the VTPN with commercial planning systems such as Varian's Eclipse through its programming interface.
This study marks the first time that intelligent treatment planning behaviors of a human planner were fully and autonomously encoded in an artificial intelligence system for external beam radiotherapy. The VTPN is not following pre-programmed rules -- it developed its own planning policy through a learning process.
The results demonstrate that deep reinforcement learning can produce treatment plans that rival those of an expert human planner, doing so consistently and in a fraction of the time. This addresses a major bottleneck in radiation oncology workflows where planning time and planner experience heavily influence patient outcomes.
The framework is designed to be generalizable. Its core architecture does not depend on the specific tumor site or treatment technique, opening the door to broad application across radiation oncology and eventually contributing to more equitable, consistent, and high-quality cancer care.