Glioblastoma (GBM), primary central nervous system lymphoma (PCNSL), and solitary brain metastasis (BM) are among the most common intracranial malignancies in adults, representing approximately 17%, 1.9%, and 14.3% of all brain tumors, respectively. Despite this frequency, they require fundamentally different treatment strategies: GBM calls for surgical resection followed by concurrent chemoradiation with temozolomide; PCNSL is treated with high-dose methotrexate-based chemotherapy without surgery; and BMs are managed depending on systemic disease status with surgery, stereotactic radiosurgery, or whole-brain radiation. Getting the diagnosis wrong before the first intervention means wrong surgery, unnecessary tissue sampling, delayed treatment, and in the case of PCNSL, the additional risk that preoperative steroid administration can cause false-negative biopsies by reducing tumor viability.
Shared MRI appearance and its consequences: On T1-weighted gadolinium-enhanced (T1Gd) MRI, all three tumor types can appear as contrast-enhancing masses with central necrosis and surrounding edema, making them difficult to distinguish based on imaging alone. GBMs typically present as iso-hypointense masses with irregular, rim-enhancing necrotic cores, but atypical GBMs may lack necrosis. PCNSLs often show homogeneous enhancement, but atypical PCNSL can develop central necrosis that mimics GBM. BMs may resemble either entity depending on the primary tumor histology. This overlap in appearances means that even experienced neuroradiologists encounter diagnostic uncertainty in a meaningful proportion of cases, and conventional imaging alone is insufficient for confident preoperative classification.
The AI opportunity: Deep learning and convolutional neural network (CNN) approaches to neuroimaging have generated considerable interest over the past decade, with studies demonstrating that these models can capture subtle texture, shape, and intensity patterns in MRI data that exceed what human observers reliably detect. The authors of this study had previously published a pilot CNN model using ResNet-101 architecture trained on 121 patients at a single institution (the Training Site, or TrS) and achieved high internal classification performance. This publication reports the external validation of that model on a separate cohort of 126 patients from a different academic center (the Testing Site, or TeS), a critical and often-omitted step before any AI diagnostic tool can be considered clinically credible.
The study follows STARD-AI reporting guidelines (Standard for Reporting of Diagnostic Accuracy Studies, Artificial Intelligence edition), which were developed as an evolution of the 2015 STARD framework to address the specific methodological requirements of AI diagnostic accuracy research. Adherence to STARD-AI is notable because many published AI diagnostic studies lack transparent reporting of the training-testing split, model selection procedures, and confidence intervals around performance metrics.
The external validation cohort consisted of 126 patients treated at Ospedale San Gerardo di Monza between January 2018 and November 2021. Eligible patients required preoperative T1Gd MRI scans and histopathological confirmation of their diagnosis. The cohort included 64 patients with GBM (mean age 64.4 years, SD 9.04), 27 with PCNSL (mean age 58.5 years, SD 16.5), and 33 with BM from various primary cancers. The BM subgroup was histologically heterogeneous: lung cancer was the most common primary (n=16, 48.4%), followed by breast cancer (n=5, 15.1%), gastrointestinal cancer (n=4, 12.1%), renal cancer (n=3, 9.1%), with additional cases from endometrial cancer and melanoma. No significant differences in age, gender, or imaging acquisition parameters were found between the three tumor groups (all p greater than 0.05), supporting a reasonably balanced cohort for external validation purposes.
Exclusion criteria: Patients were excluded if preoperative T1Gd MRI was absent or of inadequate quality per senior neuroradiologist review, if they had received prior intracranial interventions (surgery, gamma knife, or radiation), if multiple enhancing lesions were present, or in the GBM group, if IDH mutation testing was not performed. Only IDH1 and IDH2 wild-type GBMs were included, which is methodologically important because IDH-mutant gliomas have distinct imaging characteristics and a better prognosis, and their inclusion could confound classification performance.
MRI acquisition at the external site: All scans were acquired on a 1.5 T Philips Ingenia CX system. The protocol included axial T2-weighted imaging, FLAIR, diffusion-weighted imaging (DWI) with b=1000 sec/mm2, susceptibility-weighted imaging (SWI), volumetric contrast-enhanced T1Gd (Gadovist 0.1 mmol/kg), and ADC maps derived from isotropic DWI. Importantly, only the T1Gd sequence was used as input to the deep learning model, not the full multiparametric protocol, which reflects the authors' deliberate choice to work with the most widely available MRI sequence globally.
Lesion segmentation and preprocessing: Tumor regions of interest (ROIs) were manually delineated on volumetric axial T1Gd scans using the Horos DICOM Viewer (v3.3.5, open-source). After segmentation and signal intensity normalization, each ROI was centered in a 224x224-pixel black background and exported as a PNG file. This standardization to 224x224 pixels matches the input dimensions expected by the ResNet-101 architecture, which was pre-trained on ImageNet. A total of 2,853 individual axial slices were extracted: 1,748 from GBM patients (mean 28.0 per patient, SD 19.0), 412 from PCNSL patients (mean 15.0 per patient, SD 4.0), and 693 from BM patients (mean 21.0 per patient, SD 14.0).
The deep learning model is a 2D convolutional neural network based on ResNet-101 (Residual Network with 101 layers), a well-established architecture consisting of stacked three-layer residual blocks with skip connections. Skip connections address the vanishing gradient problem that plagued earlier deep networks, allowing gradients to flow directly through the network during backpropagation and enabling much deeper architectures than were previously trainable. ResNet-101 was pre-trained on ImageNet, a large natural image database, and then fine-tuned on the brain MRI dataset from the original training site (TrS, 121 patients). The pre-training on ImageNet enables the model to recognize low-level visual features (edges, textures, shapes) that transfer usefully to medical imaging tasks even though brain MRI images differ substantially from natural photographs.
Single-slice input strategy: Each extracted 2D ROI slice was treated as an independent input image, fed into all three color channels of the ResNet-101 model (which expects RGB input). This approach effectively multiplies the training and inference data, since each patient contributes multiple slices rather than a single volumetric representation. However, this also means the model makes predictions at the slice level, not directly at the patient level.
Majority voting for patient-level classification: To arrive at a final diagnosis for each patient, the model aggregated predictions across all slices from that patient using a majority-voting scheme. The predicted class for a given patient is the class most frequently assigned across all of their individual ROI slices. This ensemble-style approach within a single patient's scan helps smooth out slice-level uncertainty, particularly for heterogeneous tumors where individual slices may not be representative of the full lesion. Performance metrics were computed at the patient level, not the slice level, which is the clinically meaningful unit.
One-vs-rest (OVR) multiclass strategy: Performance metrics were extracted using a one-vs-rest (OVR) approach, where each tumor class is evaluated in a binary fashion against the combined other two classes. This generates separate AUC-ROC curves, accuracy, precision, recall, specificity, and F1-score for GBM, PCNSL, and BM individually. Bootstrap resampling (95% confidence intervals) was used to quantify statistical uncertainty around performance estimates. Python v3.7.6 with PyTorch v1.7, scikit-learn, and Seaborn were used for all modeling and analysis. Computations ran on a workstation with an Intel Core i7-10700K processor and a Tesla K80 12 GB GPU.
On the external validation dataset, the ResNet-101 model achieved the following AUC values: 0.73 (95% CI: 0.62-0.85) for PCNSL, 0.78 (95% CI: 0.71-0.87) for GBM, and 0.63 (95% CI: 0.52-0.76) for BM. These results indicate good-to-moderate discriminative performance, with GBM classification the most reliable and BM classification the weakest. The confidence intervals are notably wide for PCNSL and BM, reflecting the relatively small number of cases in those groups (27 PCNSL and 33 BM patients) and the corresponding statistical uncertainty.
Full performance metrics by class: For GBM, the model achieved accuracy 80.37% (CI: 74.8-87.0%), precision 84.13% (CI: 77.97-92.0%), recall (sensitivity) 76.14% (CI: 66.67-85.71%), specificity 84.8% (CI: 78.33-93.33%), and F1-score 0.80 (CI: 0.73-0.87). For PCNSL, accuracy was 80.46% (CI: 74.8-87.01%), precision 54.85% (CI: 44.11-70.0%), recall 66.86% (CI: 51.85-85.19%), specificity 84.29% (CI: 78.12-91.67%), and F1-score 0.60 (CI: 0.50-0.73). For BM, accuracy was 77.12% (CI: 71.54-83.74%), precision 57.71% (CI: 46.67-72.73%), recall 57.04% (CI: 42.42-72.73%), specificity 84.49% (CI: 77.78-91.14%), and F1-score 0.57 (CI: 0.45-0.70).
Patterns in the confusion matrix: The confusion matrix revealed instructive error patterns. PCNSL was misclassified nine times total; in eight of those nine errors, the model predicted BM instead of PCNSL. This directional bias is consistent with the known morphological overlap between atypical PCNSL cases that develop central necrosis and BMs, which can have similar contrast-enhancement patterns on T1Gd. Among true BM cases, the model correctly identified 19 out of 33, while the remaining 14 were split between GBM (n=8) and PCNSL (n=6). GBMs were the best-classified group overall, likely because GBM is the dominant class (64 of 126 patients) and has more recognizable irregular necrotic morphology on T1Gd.
Context for the numbers: These AUC values represent genuine external validation performance on previously unseen data from a different institution, which is inherently more stringent than the internal cross-validation AUCs typically reported in AI papers. The results demonstrate model generalizability, though with the caveat that performance was moderate rather than high for two of the three classes. The BM AUC of 0.63 is marginally above chance for a three-class problem, and the PCNSL F1-score of 0.60 indicates that many lymphoma cases are misclassified in clinical terms.
Expert neuroradiologist performance was assessed retrospectively by reviewing diagnostic reports from two neuroradiologists (P.R. and G.B.) each with at least 10 years of clinical experience. These clinicians had access to the full standard-of-care workup, including conventional MRI (T1Gd, T2, FLAIR) plus advanced sequences (diffusion-weighted imaging, susceptibility-weighted imaging, ADC maps, and in some cases dynamic contrast-enhanced or perfusion MRI), as well as clinical history, total body CT, and in some cases PET imaging. This is a critical asymmetry: the AI model operated exclusively on T1Gd images and was blinded to all other clinical information, while the radiologists drew on the full multimodal clinical picture.
Side-by-side performance: The neuroradiologists achieved F1-scores of 0.819 for GBM, 0.595 for PCNSL, and 0.818 for BM. Comparing this to the AI model's F1-scores of 0.80 (GBM), 0.60 (PCNSL), and 0.57 (BM), the AI was not inferior to the radiologists for GBM (0.80 vs. 0.819) and essentially matched them for PCNSL (0.60 vs. 0.595). The one area where the AI significantly underperformed was BM, where the radiologists' F1-score of 0.818 substantially exceeded the model's 0.57.
Accuracy comparison: Neuroradiologist accuracy was 84.09% for GBM, 82.90% for PCNSL, and 89.69% for BM. The AI model's accuracy ranged from 77-80% across all three classes, below the radiologists in each category but not dramatically so for GBM and PCNSL. The BM accuracy gap (89.69% radiologist vs. 77.12% AI) was the most clinically significant divergence.
What this comparison actually means: The "not inferior" framing for GBM and PCNSL must be interpreted carefully given the asymmetric information available to each party. The AI model achieved near-equivalent performance using only T1Gd images and no clinical context, which is notable. However, the comparison is inherently unequal: radiologists had access to ADC maps (which show diffusion restriction in high-grade lymphoma), perfusion data, and clinical context clues like steroid use history or known primary tumor. Under equal information conditions, radiologists would likely outperform the model more substantially.
The authors discuss several converging explanations for why BM classification was the weakest category. The most important is histological heterogeneity: the BM group included primaries from lung, breast, gastrointestinal tract, kidney, endometrium, and melanoma, each of which produces brain lesions with distinct radiological characteristics. A CNN trained on a pooled BM category must learn features common across these diverse primaries, which is a much harder learning problem than classifying the more histologically uniform GBM or PCNSL groups. Internal validation on the original TrS dataset had also shown lower BM accuracy (81%), meaning this was not purely a generalization failure.
The peritumoural edema problem: A key distinguishing radiological feature of BMs is abundant peritumoral edema, which reflects vasogenic edema from blood-brain barrier disruption by metastatic tumor. However, the ROI segmentation in this study was confined strictly to the T1Gd-enhancing tumor boundaries, excluding the surrounding edema zone. This deliberate methodological choice, made to avoid operator-dependent segmentation bias in the peritumoral region, likely cost the model one of its most discriminative features for BMs. Neuroradiologists, conversely, would implicitly evaluate the entire lesion context including the edema pattern.
Information asymmetry with radiologists: Radiologists also had access to whole-body CT and PET in a proportion of cases, where evidence of a known primary tumor dramatically increases the pre-test probability of BM. The AI model had no such clinical context. In real neurosurgical practice, a patient presenting with a single brain mass and a known history of lung cancer is managed very differently from a patient presenting de novo with no known systemic disease. Incorporating these clinical variables into a hybrid model could substantially improve BM classification.
Global health implications: The authors devote considerable discussion to the potential public health value of this type of tool in resource-limited settings, noting that AI-assisted diagnostics could reduce healthcare disparities in regions where expert neuroradiologists are not available. The model runs on open-source Python (PyTorch) and can be reproduced and validated at other institutions without proprietary software. Projections cited in the paper suggest AI-driven healthcare optimization in the USA alone could save up to $150 billion annually by 2026. In lower-income countries, improved mobile connectivity and cloud computing infrastructure are expanding AI feasibility even where specialty physician density remains low.
Retrospective single-scanner design: All imaging at the testing site was acquired on a single 1.5 T Philips Ingenia CX scanner. CNN models trained on images from one scanner or field strength often show degraded performance when applied to images from different hardware due to systematic differences in image contrast, noise texture, and spatial resolution. The training site used a different scanner and MRI protocol. While the fact that the model transferred reasonably well between sites is encouraging, validation on images from diverse MRI hardware, including 3 T scanners and different manufacturers, would be needed before claiming broad generalizability.
Small PCNSL and BM class sizes: With only 27 PCNSL and 33 BM patients in the external validation cohort, the confidence intervals around performance metrics for these classes are wide. The 95% CI for BM AUC (0.52-0.76) spans from near-chance to reasonable discrimination. For PCNSL precision (54.85%, CI: 44.11-70.0%), the lower bound suggests that over 45% of PCNSL predictions could be incorrect in the worst-case scenario. Larger dedicated cohorts, particularly for the rarer PCNSL class, would be needed to establish more precise performance estimates.
T1Gd-only input: The deliberate restriction to T1Gd sequences, chosen for global reproducibility, comes at a cost. Advanced sequences including ADC maps from DWI (which show characteristic diffusion restriction in PCNSL due to high cellularity), dynamic susceptibility contrast (DSC) perfusion (which reveals elevated rCBV in GBM), and SWI (which detects hemorrhagic deposits common in metastases) are well-validated discriminators in clinical practice. Including even one additional sequence such as ADC maps might substantially improve PCNSL classification specifically, since diffusion restriction is one of the most reliable imaging features of CNS lymphoma.
Offline retrospective validation only: The model has not been tested in a prospective clinical workflow. Real-time deployment introduces additional challenges including DICOM integration, segmentation automation, inference latency, clinician interface design, and the interaction between AI outputs and clinical decision-making. The performance of the model in an actual clinical environment, where it would need to operate on prospectively collected cases with variable image quality and be embedded into the diagnostic workflow, remains untested and is a critical gap before clinical adoption can be considered.
Multiparametric MRI input: The most direct improvement pathway is to extend model input beyond T1Gd to include ADC maps, T2/FLAIR, DSC perfusion, and SWI sequences. Each additional modality contributes complementary biological information: ADC captures tumor cellularity and diffusion restriction (particularly useful for PCNSL), T2/FLAIR reveals peritumoral edema extent (valuable for BM), and perfusion data distinguishes high-grade tumors with neovascularization (GBM) from low-vascularity lymphoma. Multiparametric deep learning models in brain tumor classification have generally demonstrated superior performance over single-sequence inputs, though they require more standardized acquisition protocols and larger training datasets to avoid overfitting the additional features.
3D convolutional architectures: The current model processes individual 2D axial slices and aggregates predictions by majority voting. A natural extension is 3D CNN architectures that process the entire volumetric tumor at once, capturing inter-slice spatial relationships and 3D morphological features such as sphericity, surface irregularity, and complete necrosis architecture. 3D models have shown advantages in several brain tumor classification tasks, though they increase computational requirements and typically need larger training sets to avoid overfitting given the greater number of learnable parameters.
Automated segmentation: Manual ROI delineation, as performed in this study, is not scalable to clinical deployment. Development of automated segmentation pipelines using tools such as nnU-Net or BraTS-validated segmentation models would be required for real-world use. Automated segmentation also introduces its own error sources, and the sensitivity of the classification model to segmentation quality needs to be characterized. End-to-end architectures that jointly learn segmentation and classification from weakly annotated data represent a promising direction that could reduce the annotation burden for future training datasets.
Medical education and resource-limited deployment: The authors envision this type of model serving a dual role: as a clinical decision-support tool in settings with limited neuroradiology expertise, and as an educational resource for trainees learning to recognize brain tumor morphology on MRI. The open-source codebase (Python/PyTorch) and retrospective reproducibility of the methodology are presented as assets for multi-institutional replication. Regulatory clearance, prospective multicenter validation, and integration into clinical PACS/DICOM workflows remain the critical milestones before any such tool could be deployed as a genuine decision-support system rather than a research prototype.