Accurate lung cancer staging is essential for choosing the right treatment, but existing methods have meaningful limitations. CT imaging detects mediastinal metastasis with a sensitivity of only about 55 percent, while PET-CT improves this to 77 percent but remains imperfect. Invasive approaches like mediastinoscopy offer greater accuracy but cannot be performed in all patients due to medical contraindications.
Machine learning applied to gene expression data offers a complementary approach that is objective, reproducible, and operates on molecular rather than anatomical information. Gene expression patterns captured through RNA sequencing reflect the biological state of the tumor in ways that imaging cannot directly measure, potentially revealing stage-relevant molecular signatures.
Existing machine learning models for cancer classification have struggled with interpretability, required extensive manual feature engineering, or failed to handle class imbalance between early and late-stage cases effectively. XGBoost, an optimized gradient boosting framework, addresses several of these limitations through its built-in regularization, efficient handling of structured data, and compatibility with class-weight correction.
The study used RNA sequencing data from 993 lung cancer patients in The Cancer Genome Atlas, combining the LUAD and LUSC datasets for a pan-lung cancer analysis. After excluding 32 patients with missing staging information, the final cohort was 961 patients. Patients were classified into early stage (IA, IB, IIA, IIB) or late stage (III, IIIA, IIIB, IV) for binary classification.
The dataset was randomly split 70 to 30 into training and test subsets. The Wilcoxon rank-sum test was applied exclusively to the training data to identify differentially expressed genes associated with stage, preventing information leakage from the test set into feature selection. Gene subsets ranging from the top 10 to 200 features were evaluated to identify the optimal number for the XGBoost model.
XGBoost hyperparameters including learning rate, maximum tree depth, and L1 and L2 regularization coefficients were tuned using 5-fold cross-validation within the training set. Class imbalance between the larger early-stage and smaller late-stage groups was addressed by setting the scale_pos_weight parameter to the ratio of early-stage to late-stage cases.
Model performance was assessed using area under the ROC curve on the held-out test set. Sensitivity and specificity were calculated across classification thresholds from 0.1 to 0.9 to identify operating points suitable for different clinical scenarios.
The final model using the top 40 genes achieved a test AUC of 0.6534, selected as the optimal model based on the balance between predictive performance and model simplicity. While a model using the top 140 genes achieved a slightly higher test AUC of 0.6797, it came with greater model complexity and higher risk of overfitting to the training data.
The 40-gene model showed a training AUC of 0.6106, with the test AUC of 0.6534 being notably higher than the training AUC. This pattern indicates the model is not overfitting and generalizes well to unseen data, a desirable property for potential clinical deployment where new patients will always be out-of-sample.
Threshold analysis revealed that a classification threshold of 0.3 achieved sensitivity of 77.59 percent and specificity of 36.25 percent, prioritizing the detection of late-stage cancers at the cost of more false positives. Raising the threshold to 0.4 improved specificity to 56.25 percent while sensitivity decreased to 67.24 percent, providing a better balance for scenarios where both false positives and false negatives carry clinical consequences.
Feature importance analysis ranked the 40 selected genes by their contribution to predictive accuracy as measured by the XGBoost gain metric. ECD ranked first in importance. Pathway enrichment analysis of the selected gene set identified significant involvement in cancer-related biological processes including Th17 cell differentiation, pancreatic secretion, and ribosomal function, suggesting biologically meaningful signals rather than statistical noise.
The test AUC of 0.6534 represents moderate discriminative performance, below conventional thresholds for standalone clinical deployment but sufficient to demonstrate feasibility as a complementary tool. The model's value lies in its interpretability and biological grounding, providing a starting point for further development rather than a complete replacement for existing staging methods.
The choice of classification threshold has direct clinical implications. A threshold of 0.3 maximizes sensitivity, making it appropriate in scenarios where missing a late-stage diagnosis carries the greatest harm. A threshold of 0.4 reduces false positives, making it more appropriate when unnecessary follow-up procedures for misclassified patients represent significant burden.
The dataset has meaningful limitations. The cohort is limited to patients from TCGA with confirmed diagnoses, adequate tumor purity of at least 60 percent, RNA integrity above a threshold, and sufficient nucleic acid yield, meaning the results may not directly apply to real-world clinical populations with lower quality samples. Class imbalance between early and late-stage cases also affected model training dynamics despite the weight correction strategy.
RNA sequencing itself poses practical barriers to clinical adoption, including relatively high cost and longer turnaround times compared to imaging. The authors suggest that complementary blood-based biomarkers or targeted PCR assays could offer cost-effective alternatives once the relevant gene signatures are better characterized and validated.
This study demonstrates that machine learning-driven gene selection from RNA sequencing data can produce interpretable classifiers for lung cancer staging. The 40-gene XGBoost model provides a biologically grounded, reproducible alternative perspective on tumor stage that complements rather than replaces existing imaging-based methods.
Future research should prioritize three directions: expanding the dataset to include more late-stage cases and patients from multiple institutions to reduce bias, benchmarking XGBoost against alternative models including deep learning architectures and hybrid ensemble methods, and integrating multi-omics data from proteomics and metabolomics to capture additional biological dimensions of staging.
External validation on independent cohorts is essential before any clinical translation. The optimal classification thresholds and gene signatures identified in this TCGA analysis need to be confirmed in patient populations with different demographics, treatment histories, and tissue preparation protocols to ensure robustness and generalizability.