Renal cell carcinoma (RCC) accounts for more than 90% of all kidney cancers and is the 12th most common cancer globally, with over 330,000 new cases per year. While treatments exist, including immunotherapy and targeted drugs, predicting which treatment will work best for each patient remains a major challenge because of how variable and complex the disease is.
One of the most promising sources of information for personalized medicine is gene expression data, which measures how active each of a patient's approximately 22,000 genes are at any given moment. This data can reveal molecular patterns that predict how a tumor will behave or how a patient will respond to therapy. However, analyzing tens of thousands of gene measurements for hundreds of patients creates a data complexity problem that standard statistical models struggle to handle.
This study addressed that challenge using autoencoders, a type of deep learning algorithm that compresses large, complex datasets into smaller, more manageable summaries called latent features, while preserving the most important patterns. The goal was to use these compressed summaries to predict progression-free survival (PFS), the length of time a patient lives without their cancer getting worse after starting treatment.
Data came from the JAVELIN Renal 101 trial, a phase 3 randomized clinical trial that compared two treatments: a combination of the immunotherapy drug avelumab with axitinib (a targeted therapy), versus sunitinib alone (a tyrosine kinase inhibitor, or TKI). Of 886 enrolled patients, 726 had gene expression data available, split into two cohorts of roughly equal size.
Starting with 22,955 genes, the researchers applied several filtering steps. First, they removed genes that were barely expressed in most patients. Then they used DisGeNET, a scientific database that links genes to diseases, to keep only genes known to be associated with kidney cancer. This reduced the gene set to 2,403 relevant genes per patient.
In addition to tabular gene data, the team built a protein-protein interaction (PPI) network specific to kidney tissue, based on data from the PPT-Ohmnet platform. In this network, each gene is a node and physical interactions between proteins are the connections. This allowed a second type of model, called a graph autoencoder, to consider not just individual gene levels but also how genes influence each other through known biological pathways.
An autoencoder works in two phases: an encoder squeezes high-dimensional data into a small set of latent features, and a decoder tries to reconstruct the original data from those features. By minimizing the reconstruction error, the autoencoder learns which patterns in the data are most essential. The team tested four variants: standard, denoising (which adds artificial noise to improve robustness), sparse (which keeps only the most active features), and variational (which makes the compressed space smooth and continuous).
A graph autoencoder extends this concept by incorporating the PPI network. Rather than treating each gene independently, the model considers the connections between proteins, allowing biological context to shape how features are compressed. For each of the 726 patients, the researchers built an individual graph with the same structure but different node values reflecting that patient's gene expression profile.
After compression, the latent features were fed into a Cox Proportional Hazards (COX PH) model, a standard statistical method for survival analysis, combined with Breslow's estimator to generate individual survival probability curves over time. The pipeline was evaluated through 10-fold cross-validation to ensure results were reliable across different patient subsets.
Among tabular autoencoders, the denoising autoencoder reconstructed gene expression data most accurately. However, for predicting patient survival (measured by time-dependent AUC under the ROC curve), the sparse autoencoder performed best, achieving an area under the ROC of 0.71 in the sunitinib arm. The variational autoencoder performed worst, likely because constraining the latent space to a specific distribution caused information loss.
The graph autoencoders did not reconstruct raw gene data as well as tabular autoencoders, but a combination of sparse and denoising penalties achieved the best PFS prediction loss (3.45) in the TKI arm. This suggests that protein interaction context adds value for survival prediction even when reconstruction quality is lower.
A notable finding was that predictions were more accurate for the sunitinib arm than for the immunotherapy arm. The immunotherapy predictions performed close to random, which may reflect the greater biological complexity and variability of immune responses. Additionally, all models tended to overestimate how long patients would remain progression-free, a known limitation of Breslow's estimator with small sample sizes.
To understand what the models were learning, the team analyzed which genes were most strongly associated with the five latent features most valued by the Cox model. Using mutual information (a measure of how much knowing one variable tells you about another), they identified five genes that appeared most often across folds: LRP2, NAT8, ACE2, CYP4A11, and EMX2.
LRP2 (megalin) is a receptor found on kidney tubule cells that is involved in protein reabsorption and signaling. ACE2 is well-known for its role in blood pressure regulation and has more recently gained attention for its role in cell entry and cancer biology. NAT8 is involved in kidney cell metabolism. All five genes had previously been linked to RCC prognosis in the scientific literature, validating that the autoencoder was extracting biologically meaningful patterns.
This interpretability component is particularly important because one of the criticisms of deep learning models in medicine is that their reasoning is opaque. By using mutual information to trace which original genes most influenced the model's compressed representations, this study demonstrates a way to make autoencoder-based predictions more transparent and scientifically informative.
This research demonstrated that autoencoders are well-suited for handling the complexity of transcriptomic data in cancer survival analysis. Compared to traditional dimensionality reduction like PCA, the non-linear capabilities of autoencoders led to approximately 36% better survival predictions. This positions autoencoders as a valuable tool for translating large gene datasets into actionable clinical insights.
The finding that different autoencoder types excel at different tasks is an important practical lesson: denoising autoencoders are best for faithfully reproducing gene expression patterns, while sparse autoencoders produce more clinically meaningful compressed representations. Combining penalties can capture the strengths of both approaches.
Future work should focus on expanding the patient dataset, which was a limiting factor in this study, and exploring alternative survival estimators that are less prone to overestimating how long patients will remain progression-free. There is also promise in applying this framework to other cancers and other types of omics data, working toward AI-guided personalized treatment selection for patients facing complex diseases.