Breast cancer treatment decisions are heavily influenced by axillary lymph node status - whether cancer has spread to the lymph nodes under the arm. Lymph node positivity typically prevents de-escalation of systemic therapy and often mandates lymph node radiotherapy, making accurate staging essential for selecting the right treatment intensity.
An increasing proportion of patients are treated with neoadjuvant systemic therapy (NST) - chemotherapy or targeted therapy given before surgery. NST assignment is based on tumor biology and radiological findings, but existing imaging methods misdiagnose axillary node status in up to 30% of patients, potentially misdirecting treatment selection.
After NST, initial node status becomes even harder to determine because the treatment can eliminate metastatic cells from nodes that were originally positive. This creates uncertainty about treatment intensity for HER2-positive patients, for example, regarding whether to continue dual anti-HER2 blockade. Non-invasive tools that predict preoperative lymph node status more reliably are therefore a clinically urgent need.
This study trained and evaluated multiple machine learning models on clinicopathological data from a national Croatian breast cancer registry, with the primary goal of predicting axillary lymph node metastasis without imaging or invasive procedures. The analysis was designed specifically around patients meeting criteria for NST - the group where accurate preoperative staging has the highest clinical impact.
The study is distinctive for its scale - using data from 8,381 patients collected across all Croatian hospitals over five years - and for its focus on model explainability. Rather than treating the models as black boxes, the authors used SHAP (SHapley Additive exPlanations) values to show which clinical features drove each prediction and whether the patterns the models learned align with established clinical knowledge.
Three algorithms were compared: logistic regression (as a transparent baseline), random forest, and XGBoost. The study reported separate results for the NST-criteria subgroup (719 patients) and the entire population (8,381 patients), enabling direct comparison of model performance in the target clinical population versus a general breast cancer cohort.
Data were collected from all Croatian hospitals treating breast cancer over a five-year period (January 2017 to January 2022). After removing entries with missing values and excluding the 1,324 patients who received neoadjuvant therapy (to avoid the confounding effect of NST on node status), the clean dataset contained 8,381 patients who underwent primary surgery.
The NST-criteria subgroup was defined using NCCN guidelines: all tumors larger than 5 cm regardless of subtype, tumors of 2 cm or larger that were triple-negative or HER2-positive, and inflammatory breast cancers. This yielded 719 patients - those who would typically be considered for NST in current clinical practice. This subgroup had a notably balanced target variable: 55% node-positive versus 45% node-negative, unlike the broader population (34% positive vs. 66% negative).
Ten clinical features were used: age, tumor size, histological type, immunophenotype, grade, ER and PR receptor levels, HER2 levels, and Ki-67 proliferation index. All models were trained on 80% of data using stratified 5-fold cross-validation with grid search hyperparameter optimization, then evaluated on the held-out 20% test set. Bootstrap resampling (2,000 samples) was used to generate confidence intervals for all performance metrics.
In the NST-criteria group, random forest achieved the best performance after hyperparameter optimization, with an AUC of 0.793 (95% CI: 0.713-0.865), sensitivity of 0.809, and specificity of 0.570. XGBoost followed closely with an AUC of 0.783, while logistic regression reached 0.763. A univariate baseline model using only tumor size achieved AUC 0.645, confirming that multi-feature models meaningfully outperform single-predictor approaches.
In the total study population of 8,381 patients, XGBoost performed best with AUC 0.762 (95% CI: 0.726-0.795), closely followed by random forest at 0.760. Logistic regression reached 0.741. All models performed somewhat lower in the total population than in the NST group, which the authors attribute to the total population's greater heterogeneity and class imbalance (only 30% node-positive).
For the XGBoost model in the total population, adjusting the decision threshold from 0.5 to 0.28 (to maximize F1 score) substantially changed the performance profile: sensitivity increased from 0.344 to 0.732 and negative predictive value from 0.761 to 0.854, at the cost of reduced specificity (from 0.903 to 0.676). This threshold flexibility is clinically relevant - clinicians can adjust the operating point based on whether missing a positive case or over-treating a negative case is the greater concern.
In the NST-criteria group, SHAP analysis identified tumor size as the dominant predictor of lymph node metastasis, followed by ER, PR, and HER2 status. The relationship between tumor size and nodal involvement was approximately linear up to 5 cm, after which it reached a plateau - consistent with published evidence from Sopik and Narod showing the same saturation effect.
In the total study population, tumor size remained the most important feature, but the second most important predictor shifted to Ki-67, followed by age and tumor grade. The models revealed a non-linear relationship for Ki-67: values above 25% were associated with higher metastasis risk, but values above 75% were associated with lower risk - an effect the SHAP analysis attributed to the triple-negative immunophenotype, which tends to have very high Ki-67 but different metastatic patterns than luminal subtypes.
Age showed a non-linear effect across the full population: both patients younger than 40 and patients older than 75 were associated with higher nodal metastasis rates - a finding consistent with prior clinical literature. For HER2, positivity was associated with higher metastasis probability across both analyses. Importantly, SHAP values independently identified the standard clinical cutoff points for ER, PR, Ki-67, and HER2 positivity, validating that the model was learning genuine biological relationships rather than dataset artifacts.
The study's finding that random forest and XGBoost outperformed other approaches reflects a broader pattern in machine learning research: tree-based ensemble models consistently outperform deep learning on tabular clinical data, particularly when training samples number in the thousands rather than millions. The authors cite published evidence that tree-based models outperform neural networks on tabular datasets with up to 10,000 training examples.
Compared to other studies using only clinicopathological features, this study's AUC of 0.79 (NST group) compares favorably. A comparable neural network model achieved only 0.72 (Zheng et al.), and another neural network study reached 0.74 (Dihge et al.). The study argues this relative advantage of tree-based approaches over neural networks for structured clinical data - even with larger datasets - is an important practical consideration for deploying models in resource-constrained healthcare settings.
A key limitation is that the study used post-surgical pathology to define the ground truth lymph node status, which differs from the preoperative setting where the model would be deployed. Tumor size from imaging (rather than surgical specimen) was used where available, and the authors argue concordance between imaging and pathologic measurements is high enough for the model to generalize. Single-country data from Croatia also limits direct generalizability to other populations.
The practical value of this model is greatest for patients being evaluated for neoadjuvant treatment, where imaging-based lymph node assessment is least reliable. An AI tool that predicts lymph node positivity from routine pre-biopsy clinical data could supplement imaging findings, potentially identifying patients who are actually node-positive but appear node-negative on ultrasound - and thus should be directed to NST rather than primary surgery.
Beyond treatment selection, the model could reduce reliance on sentinel lymph node biopsy for staging in patients where the model confidently predicts node-negative status. The false-negative rate of sentinel node biopsy itself reaches up to 27% in some studies, meaning AI-based prediction as a complementary tool could improve overall staging accuracy even compared to surgical methods.
The authors propose that locally validated versions of such models could be integrated into multidisciplinary team workflows at the point of treatment planning. Future improvements could incorporate genetic expression data and serum biomarkers - features not included in this analysis - which could increase predictive accuracy and enable even more precise treatment individualization.
This study demonstrates that tree-based machine learning models trained on routinely available clinicopathological features can achieve meaningful performance in predicting breast cancer lymph node metastasis, with AUC of 0.793 in the NST-eligible population - comparable to or exceeding prior studies that incorporated imaging or more complex data types.
A key contribution is the use of SHAP explainability to verify that the models learned clinically valid relationships rather than spurious correlations. The confirmed alignment between model-learned feature importances and established clinical knowledge strengthens the case for clinical adoption, as clinicians can audit the model's reasoning rather than accepting opaque predictions.
Future directions include multinational multicenter validation to establish generalizability beyond the Croatian cohort, and integration of genetic and biomarker data to improve model performance. The authors envision locally optimized versions of these models being deployed as decision support tools in breast cancer multidisciplinary team meetings.