The androgen receptor (AR) is the primary drug target in prostate cancer. It is a protein that, when activated by male hormones (androgens), drives cancer cell growth. A class of drugs called anti-androgens (or AR inhibitors) is the cornerstone of prostate cancer treatment, blocking the receptor to stop tumor growth.
Unfortunately, resistance to these drugs almost always develops eventually. One major mechanism is through gain-of-function mutations -- genetic changes in the AR gene that alter the receptor's shape so that drugs can no longer block it. In some cases, the mutation actually flips the drug's effect, turning it from a blocker into an activator that promotes cancer growth instead of stopping it.
This makes identifying resistance-causing mutations a clinical priority. If doctors know which mutations confer resistance before starting treatment, they can choose different drugs or monitor patients more closely. Conversely, drug designers need to know which mutations to watch for so they can build compounds that avoid triggering them.
Experimentally testing every known AR mutation against every drug in lab cells is time-consuming and expensive. This study asks whether a deep neural network (DNN) trained on existing experimental data can accurately predict how any given AR mutation will respond to a given anti-androgen drug.
The training data came from experimental measurements of 68 distinct AR mutants, either identified in prostate cancer patient blood samples or documented in scientific literature. Each mutant was tested against seven different anti-androgen drugs, including bicalutamide, enzalutamide, apalutamide, and darolutamide, across 11 increasing drug concentrations using a luciferase reporter assay in PC3 prostate cancer cells.
Rather than a simple active/inactive binary label, drug responses were classified into four categories: antagonist (drug blocks the receptor as intended), agonist (drug activates the receptor -- resistance), mixed-response (partially blocked but with residual activation -- a form of resistance), and non-responsive (the mutant receptor is non-functional and does not respond to anything).
Because the resistant and agonist phenotypes are rare in real patients, there was a significant class imbalance in the dataset. To prevent the model from simply predicting the most common outcome, the authors used a technique called borderline SMOTE (Synthetic Minority Over-sampling Technique) to artificially create additional examples of the rare classes during training.
The data were split 80% for training and cross-validation and 20% for final testing, with stratified sampling to ensure all drugs appeared in both sets.
A key innovation in this study is that neither the 3D structure of the protein nor docking simulations were needed. Instead, the model used purely sequence-based protein descriptors. Each amino acid in the AR sequence was encoded using five numerical values describing its physical and chemical properties (z-scale descriptors), resulting in a compact and generalizable representation.
Drug molecules were represented as Morgan fingerprints (also called extended connectivity fingerprints, ECFP), a standard method in computational chemistry. Each drug is encoded as a 2048-bit binary vector where each bit indicates whether a particular structural fragment is present or absent in the molecule.
This approach -- using protein sequence descriptors combined with chemical fingerprints -- is called proteochemometrics. It allows the model to simultaneously consider both the identity of the mutated protein and the chemical structure of the drug, predicting how they will interact without requiring expensive structural calculations.
The DNN itself had two hidden layers (128 neurons and 32 neurons), used ReLU activations, dropout regularization to prevent overfitting, and was optimized using the Adam optimizer with early stopping. This relatively compact architecture was appropriate given the limited number of training examples.
The DNN was benchmarked against two standard machine learning methods: support vector machines (SVM) and random forest (RF). While all three methods showed similar performance on cross-validation metrics, the DNN significantly outperformed both baselines on the held-out test set.
The DNN achieved a weighted average precision, recall, and F1 score of 0.80, 0.79, and 0.79 respectively. The random forest scored 0.75 across all three metrics, and the SVM scored only 0.62. The DNN also had the highest Matthews Correlation Coefficient (MCC) at 0.654 versus 0.601 for RF and 0.336 for SVM -- a metric that is particularly informative for imbalanced class problems.
The key advantage of the DNN emerged in distinguishing between subtle phenotypes. Both SVM and random forest tended to over-predict the antagonist (majority) class and performed poorly at identifying the mixed-response and non-responsive phenotypes. The DNN was significantly better at discriminating between these clinically important but less common outcomes.
The confusion matrices revealed that the DNN correctly identified 86% of non-responsive mutants, while both baseline methods missed a substantial fraction of these cases -- an important clinical distinction, since non-responsive mutants may indicate that treatment can continue while resistance mutants require switching drugs.
The most compelling part of the study was a prospective validation experiment. The trained DNN was used to predict responses of 44 AR mutants to darolutamide, a newer anti-androgen drug that had not been included in training. These mutants had been tested with older drugs (bicalutamide, enzalutamide) but never with darolutamide.
The model predicted that 31 of the 44 mutants would be antagonized by darolutamide (drug works), 12 would be non-responsive, and just one mutant (E666D) would show a resistant mixed-response phenotype. The authors then tested all 44 mutants in the lab with darolutamide to check these predictions.
The experimental results matched the model's predictions almost exactly. Of the 44 mutants, 43 were correctly classified. The single incorrect prediction (E666D) was predicted to be resistant but tested as antagonized in the cell assay, though minor activation was observed at high drug concentrations, suggesting possible borderline resistance.
Notably, the baseline RF and SVM methods performed poorly on this validation set -- RF predicted 13 resistant mutations (when the true number was 0) and SVM predicted 0 non-responsive cases (when the true number was 12). This confirms the DNN's superior ability to generalize to new data beyond the training set.
The finding that darolutamide antagonized nearly all 44 tested AR mutants without triggering resistance is itself clinically significant. Unlike older drugs (bicalutamide, apalutamide, enzalutamide), which share a nearly identical core chemical structure and have shown similar resistance patterns, darolutamide has a structurally distinct chemical scaffold that may reduce its tendency to activate resistance mutations.
In the clinic, this model could serve as a resistance screening tool. When a patient's tumor is sequenced and AR mutations are identified, the model could predict whether those specific mutations are likely to resist current drugs, helping physicians choose the most effective therapy upfront rather than waiting for treatment failure.
During drug development, the model could be applied to screen candidate anti-androgen compounds early in development, flagging those most likely to elicit resistance mutations before they advance to expensive clinical trials. This could shift drug design toward structurally diverse compounds less prone to resistance.
The approach is also generalizable. The proteochemometric framework used here -- combining protein sequence descriptors with chemical fingerprints -- is not limited to the androgen receptor or prostate cancer and could be adapted for other cancer-driving proteins that develop drug resistance mutations.
This study demonstrates that a deep neural network using proteochemometric descriptors can accurately predict how acquired androgen receptor mutations will respond to anti-androgen drugs, outperforming standard machine learning methods in both overall accuracy and discrimination of rare resistance phenotypes.
The prospective validation against darolutamide, where predictions were confirmed in live cell experiments, represents a strong proof of concept for the real-world predictive power of this approach.
Future work will expand the dataset to cover more mutants and combination mutations, which are common in advanced prostate cancer. As more patient mutation data accumulates from tumor sequencing studies, the model's accuracy and coverage are expected to improve further.
All code and datasets are publicly available, enabling other research groups to apply this framework to other drug-resistance problems in cancer and beyond.