Colonoscopy is one of the most powerful tools doctors have for preventing and detecting colorectal cancer, the third most common cancer in Canada. When performed on schedule, colonoscopies can find and remove pre-cancerous polyps before they become cancer, and detect cancer early when it is most treatable.
However, the COVID-19 pandemic caused a dramatic drop in colonoscopy and other endoscopy volumes across the country, creating a large backlog of patients waiting for these procedures. Modeling studies have shown that if this backlog is not addressed aggressively, it could lead to excess deaths from colon cancer for many years to come.
Even before the pandemic, wait times for colonoscopy were already a challenge due to limited resources and high demand. Solving this access problem is therefore not just a matter of convenience but a genuine public health priority with direct implications for colorectal cancer outcomes.
Currently, most endoscopy units give every patient the same flat appointment slot of 30 minutes, regardless of the procedure type or patient characteristics. In reality, actual procedures range from less than 5 minutes to over an hour, depending on many factors.
This one-size-fits-all scheduling approach wastes time. When short procedures are given 30-minute slots, there is idle time between cases. When complex procedures run long, the schedule falls behind, causing delays for later patients and staff overtime.
If the endoscopy unit could accurately predict how long each procedure would take before scheduling it, appointments could be sized to fit, squeezing more patients into the same working hours. This study aimed to build exactly that kind of machine learning prediction and scheduling system.
The research team gathered data on 1,332 endoscopy procedures performed at two major hospital sites in Montreal before the pandemic. For each procedure, they recorded patient characteristics such as age, sex, body mass index, medical history, type of procedure, and reason for the procedure.
They then tested several machine learning approaches to predict how long each procedure would take, including Linear Regression, Random Forest, and a method called XGBoost (Extreme Gradient Boosting). The models were trained on 80 percent of the data and tested on the remaining 20 percent to ensure the results would hold up on new cases.
The team also used a systematic feature selection process to identify which patient characteristics most strongly predicted procedure length. They found that only four key features were needed: procedure type (such as colonoscopy versus gastroscopy), endoscopist type, the presence of respiratory disease, and the presence of type 2 diabetes.
Once procedure duration could be predicted for each patient, the team built a scheduling system to use those predictions to fill the schedule more efficiently. They developed two approaches: a simpler greedy heuristic and a more sophisticated tool called a Mixed Integer Programming (MIP) model.
The greedy heuristic sorted patients by priority and due date, then assigned each one to the earliest available slot. The MIP model took a more mathematical approach, treating the scheduling problem as an optimization equation that simultaneously maximized the number of patients seen and minimized how late high-priority patients had to wait.
Both approaches were tested in simulation on a hypothetical waiting list of 1,000 patients, and also on a set of real patient data from the hospitals, covering an eight-day scheduling window. The simulations were designed to check not only whether more patients could be seen, but also whether the AI-sized appointments would create problematic levels of overtime for staff.
The XGBoost model predicted procedure duration with a mean absolute error of only 5.67 minutes, meaning its guesses were typically within about 5 to 6 minutes of the actual procedure length. This level of accuracy was better than all other tested models.
In the simulation with 1,000 patients, the standard fixed 30-minute schedule could book 780 patients in the two-week period. The MIP approach scheduled 826 patients, an increase of 5.9 percent, and reduced average waiting time from 19.5 days to approximately 17.3 days.
When tested on real patient data over eight working days, the MIP model scheduled 8 more patients than the fixed 30-minute baseline while keeping overtime reasonable at about 20 minutes per room per day. The fixed-slot schedule, paradoxically, also generated overtime (about 13 minutes per day) despite leaving idle gaps, showing how poorly matched the flat schedule was to real procedure times.
For patients and families worried about colorectal cancer, long waits for colonoscopy can be frightening. Research has shown that delays between a positive stool test and colonoscopy are associated with higher cancer risk and more advanced disease at diagnosis. Getting the colonoscopy done on time genuinely matters.
A scheduling tool that can fit approximately 6 percent more patients into the same working days could meaningfully reduce how long patients wait. Over a waiting list of thousands of patients, this adds up to many more people getting timely diagnoses and cancer prevention.
The system could also be particularly valuable for high-priority patients, such as those with symptoms or abnormal screening tests. The MIP model was specifically designed to prioritize these patients, reducing their overdue days more aggressively than lower-priority surveillance cases.
The data for this study came from two hospitals in Montreal collected before the pandemic, which may limit how well the model generalizes to other centers or to the post-pandemic environment where patient characteristics and procedure mixes may have changed.
Because the study relied on historical data with manually recorded timestamps, the dataset was smaller than ideal, with some missing data. The authors note that prospective data collection with properly designed electronic recording would improve model accuracy and allow the system to be refined over time.
The simulation also did not account for patient no-shows, which are a real-world factor that can create unplanned gaps in the schedule. Future implementation would need to address no-show rates and staff resilience to ensure the system works smoothly in clinical practice.
The next step is a real-world clinical trial to confirm that the benefits seen in simulation actually translate to patient throughput gains when the tool is deployed in a live endoscopy unit.
This study demonstrates that combining a machine learning prediction model with an optimization-based scheduling tool can meaningfully increase how many patients receive endoscopy procedures in a given period, reducing both waiting times and the risk that people miss timely colorectal cancer screening.
The approach is practical because it uses data that is already routinely collected in hospitals: patient age, medical history, procedure type, and indication. No new data collection infrastructure is required. The XGBoost model is also computationally efficient and interpretable, qualities that matter for clinical adoption.
By treating the endoscopy backlog as a mathematical optimization problem, this research provides a concrete, evidence-based solution to a health system challenge with direct consequences for colorectal cancer prevention and early detection in the population.