Machine learning to identify patients at risk of developing pancreatic cancer in primary care: a retrospective case-control study

PLoS One 2021 AI 7 Explanations View Original
Original Paper (PDF)

Unable to display PDF. Download it here or view on PMC.

Plain-English Explanations
Pages 1-2
The Urgent Need for Earlier Pancreatic Cancer Detection

Pancreatic cancer is one of the deadliest cancers precisely because the vast majority of cases are diagnosed when the disease has already spread beyond the pancreas. Only about 15 to 20 percent of patients present with surgically resectable disease, and the five-year survival rate across all stages remains below 12 percent.

The key to improving outcomes lies in identifying patients before symptoms develop, when the tumor is still small and localized. Unlike breast or colorectal cancer, pancreatic cancer lacks an established population-level screening program, partly because the disease is not common enough in the general population to make broad screening cost-effective.

A more targeted approach would be to identify the subset of primary care patients at elevated risk months or years before their cancer diagnosis, focusing surveillance resources where they would do the most good. Primary care records contain rich longitudinal data including diagnoses, medications, laboratory results, and symptoms that may reflect early cancer signals.

This study used machine learning applied to real-world primary care data from the United Kingdom to develop a risk prediction model capable of flagging patients likely to be diagnosed with pancreatic cancer within the next 20 months.

TL;DR: Pancreatic cancer is almost always diagnosed too late; this study used machine learning on UK primary care records to identify high-risk patients months before their diagnosis would normally occur.
Pages 2-4
Study Design: UK Primary Care Data and Case-Control Matching

The study used data from the Clinical Practice Research Datalink (CPRD), one of the world's largest primary care databases, linked to the National Cancer Registration and Analysis Service (NCRAS) to confirm cancer diagnoses. This linkage allows primary care records to be matched with confirmed cancer registry outcomes.

A retrospective case-control design was used. A total of 1,139 confirmed pancreatic cancer cases were identified and matched to cancer-free controls from the same primary care population. Matching was based on age, sex, and practice to ensure comparable baseline populations.

The analysis focused specifically on patients under 80 years of age at diagnosis, with an index date defined as 20 months before the cancer diagnosis date for cases, and an equivalent date for matched controls. This 20-month window was chosen to represent a realistic lead time for clinical intervention.

Features used to train the models were extracted from primary care records in the period before the index date and included over 150 clinical variables: diagnoses, medications, laboratory test results, and symptom codes. All features reflected information that would have been available to a GP at the index date, ensuring the model reflects real clinical decision-making conditions.

TL;DR: Using UK CPRD data linked to cancer registry records, 1,139 pancreatic cancer cases were matched to controls and ML models trained on primary care variables from 20 months before diagnosis.
Pages 4-5
Machine Learning Models Used: Logistic Regression and Random Forest

Two machine learning approaches were compared: logistic regression, a traditional statistical classification method that provides interpretable linear relationships between features and risk, and random forest, an ensemble method that builds multiple decision trees and averages their predictions to capture non-linear interactions.

Both models were trained on the same feature set derived from primary care records. Feature selection was performed to identify the most predictive variables and reduce dimensionality, helping prevent overfitting on the relatively small case dataset.

Model performance was evaluated using standard metrics including sensitivity (proportion of true cases correctly identified), specificity (proportion of true non-cases correctly identified), and area under the ROC curve (AUC), which summarizes the discrimination ability of the model across all possible classification thresholds.

Cross-validation procedures were applied to estimate out-of-sample performance and assess how reliably the models would generalize to new patients not seen during training. This is an important methodological requirement for any predictive model intended for clinical use.

TL;DR: Logistic regression and random forest models were trained and cross-validated on over 150 primary care variables, with performance evaluated using sensitivity, specificity, and AUC metrics.
Pages 5-7
Model Performance: Identifying Patients 20 Months Before Diagnosis

The best-performing model identified 41.3% of pancreatic cancer cases in patients under 80 years old, with a 20-month lead time before clinical diagnosis. This means that nearly half of eventually-diagnosed patients would have been flagged by the algorithm nearly two years before their cancer became apparent through conventional clinical presentation.

At the optimal operating threshold, the model achieved a sensitivity of 72.5% and a specificity of 59%, with an overall AUC of 0.66. While an AUC of 0.66 is modest by some standards, in the context of a disease as rare and difficult to detect as pancreatic cancer, this level of discrimination represents clinically meaningful risk stratification.

The random forest and logistic regression models performed comparably overall, suggesting that the predictive signal in the data is largely captured by linear relationships between features and cancer risk, without requiring the non-linear modeling capacity of ensemble methods.

Sensitivity varied by age group, with the model performing somewhat better in younger patients (under 65). This may reflect that younger patients have fewer confounding comorbidities, making pancreatic cancer-related signals more distinguishable in their records.

TL;DR: The model identified 41.3% of eventual pancreatic cancer patients 20 months before diagnosis, with 72.5% sensitivity and 59% specificity, representing a meaningful early detection signal in primary care data.
Pages 6-8
Diabetes as the Principal Risk Factor

New-onset diabetes mellitus emerged as the most important risk factor in the predictive models. The relationship between diabetes and pancreatic cancer is bidirectional and complex: diabetes can arise as a consequence of pancreatic cancer (cancer-associated diabetes) up to two or three years before diagnosis, or pre-existing type 2 diabetes can increase cancer risk over time.

The model captured the significance of recent diagnosis of diabetes as a particularly strong signal, distinguishing it from long-standing diabetes which is less specifically predictive. A new diabetes diagnosis in a middle-aged or older adult with no previous metabolic risk factors may reflect early pancreatic dysfunction caused by an occult tumor.

Other significant predictors included weight loss, abdominal pain, dyspepsia, and jaundice, reflecting the symptomatic constellation of pancreatic cancer. However, these symptoms are often present only when disease is already advanced, highlighting the importance of pre-symptomatic risk markers like diabetes.

Laboratory abnormalities, including elevated liver enzymes and changes in blood glucose trajectory, also contributed predictive value. The ability of the algorithm to integrate multiple subtle signals simultaneously is a key advantage over any single clinical test or biomarker.

TL;DR: New-onset diabetes was the strongest predictor in the model, reflecting cancer-associated pancreatic dysfunction emerging months to years before diagnosis, alongside weight loss, abdominal symptoms, and lab abnormalities.
Pages 8-10
How This Could Change Primary Care Practice

If deployed in primary care, a validated risk prediction tool like this could function as a clinical decision support system running continuously on electronic health records, automatically flagging patients whose accumulated risk factors cross a defined threshold. GPs could then be alerted to consider further investigation, such as CA 19-9 blood testing or pancreatic imaging.

The model is designed to work on data that is already routinely collected in primary care without requiring any new tests or patient actions. This makes implementation feasible in existing healthcare systems without changing clinical workflows significantly.

Even a 41% detection rate 20 months early, while not capturing the majority of cases, could still represent a substantial improvement in outcomes. In pancreatic cancer, the difference between stage I and stage III diagnosis is the difference between a realistic chance of cure and almost certain death within two years. Earlier identification of even a subset of patients has major clinical value.

The approach could also help prioritize patients for emerging liquid biopsy or other blood-based cancer detection tests, using the ML risk score as a pre-screening layer to concentrate high-sensitivity testing on the highest-risk individuals, improving the cost-effectiveness of such programs.

TL;DR: Deployed as a clinical decision support tool, this ML model could trigger earlier investigation in flagged patients using only routine primary care data, potentially shifting diagnoses toward earlier and more treatable stages.
Pages 10-11
Limitations and the Path to Clinical Validation

The study acknowledges several important limitations. The retrospective design means the model was developed and tested on historical data, and performance may differ prospectively when deployed in real primary care settings where data quality, coding practices, and patient populations may vary.

The AUC of 0.66 indicates a modest but real predictive signal. For a model to be clinically deployed, further validation in independent prospective cohorts across different geographic and demographic settings is required. The model must also be tested for health equity, ensuring it does not underperform for specific ethnic or socioeconomic groups.

The study represents an important proof of concept that machine learning applied to routine primary care data can identify pancreatic cancer risk with meaningful lead time. Future iterations should incorporate additional data types such as genomic risk scores, imaging results, and wearable device data to improve performance.

Prospective clinical trials testing whether algorithmic flagging followed by systematic surveillance actually improves pancreatic cancer outcomes are the essential next step. The authors call for such trials and for investment in infrastructure to enable GP-level algorithmic risk screening in routine clinical practice.

TL;DR: The study is a promising proof of concept but requires prospective validation across diverse populations; future iterations incorporating genomic and imaging data could substantially improve the model's clinical utility.
Citation: Open Access, 2021. Available at: PMC8171946.