Diffuse large B-cell lymphoma (DLBCL) is the most common subtype of non-Hodgkin lymphoma (NHL) worldwide, and it is a highly heterogeneous malignancy. Standard first-line treatment with R-CHOP (rituximab, cyclophosphamide, doxorubicin, vincristine, and prednisone) cures approximately 50-60% of patients, but 40-50% experience either primary refractory disease or early relapse. This subgroup, collectively referred to as relapsed or refractory (R/R) DLBCL, carries a significantly worse prognosis than patients who achieve durable remission.
The clinical landscape of R/R disease: Among patients who develop R/R disease, the breakdown is clinically important. Approximately 15-25% experience progression during or immediately after treatment (primary refractory disease), 20-30% relapse after achieving complete remission (CR), and roughly 5% progress after achieving only partial remission (PR). Patients in the R/R category have meaningfully worse survival than non-R/R patients, and salvage therapies carry their own burdens. Among patients who receive second-line intensive treatment, only 27% respond and proceed to autologous stem cell transplantation (ASCT), with a median overall survival of only 10 months.
Why current tools fall short: Several biomarkers have been explored to predict R/R risk, including MYC/BCL2 rearrangement (double-hit lymphoma), cell-of-origin (COO) classification into germinal center B-cell (GCB) versus activated B-cell (ABC) subtypes, and the International Prognostic Index (IPI). However, no existing model combines clinical features, molecular pathological data, laboratory test results, and H&E pathological images into a unified predictive framework. Prior to this study, there was specifically no model predicting R/R risk from routine H&E whole slide images (WSI).
The motivation for early R/R identification is practical and urgent: if high-risk patients can be flagged at diagnosis, clinicians can consider intensified induction regimens, enroll patients in clinical trials earlier, or proactively plan for CAR-T cell therapy, which is approved as second-line therapy for high-risk large B-cell lymphoma and may yield better outcomes when applied earlier rather than after multiple salvage lines have failed.
The study cohort was drawn from the Affiliated Hospital of Xuzhou Medical University, collecting all DLBCL patients diagnosed between 2015 and 2018. After applying inclusion and exclusion criteria, 227 patients met the enrollment requirements. The median follow-up duration was 45.5 months. All diagnoses complied with the 2016 World Health Organization classification of lymphoid neoplasms, and each patient had a biopsy-confirmed DLBCL diagnosis with complete clinical information, follow-up data, and histological specimens.
Inclusion and exclusion criteria: Patients required a minimum follow-up of 36 months after completion of treatment, complete clinical records, and available H&E samples from surgical biopsies only (puncture specimens without wax blocks or with insufficient tumor cell content were excluded). Cases involving transformation from other lymphoma subtypes were excluded, as were patients with other active malignancies or severe cognitive or psychiatric impairment. The TCGA DLBCL database was evaluated as a potential external validation source but was found to have incomplete follow-up duration and insufficient clinical detail to determine R/R status reliably, so it was not used.
Patient characteristics: Of the 227 patients, 53.74% were male and 52.87% were 60 years or older. A majority (59.91%) were diagnosed at stage I-II. Chemotherapy regimens included CHOP-like (57.27%), EPOCH-like (20.70%), CVAD-like (7.05%), and MTX-based (4.41%) approaches. Only 45.37% received rituximab during first-line treatment. Among outcome categories, 44.05% had progressive disease (PD) as their best response, while 26.43% achieved complete remission (CR) and 17.62% achieved partial remission (PR).
R/R group definition: The 227 patients were divided into two groups based on treatment response. The R/R group comprised 118 patients (51.98%), of whom 47 were classified as refractory (20.70%) and 71 as relapsed (31.3%). Refractory disease was defined as failure to achieve remission at any point during treatment, or progression less than one month after achieving PR, or relapse within one month after CR. The non-R/R group comprised 109 patients.
The study built three separate predictive models, then combined two of them into a fusion model. The dataset was split into 70% training, 10% validation, and 20% testing sets, stratified to maintain the R/R-to-non-R/R ratio across all splits. To minimize random errors from any single partition, 10 independent repeat trials were conducted for each model, and AUROC values are reported as means with standard deviations across those trials.
H&E image preprocessing: High-resolution (20x magnification) whole slide images (WSI) were collected for all enrolled patients. Each WSI contains both tissue and white background (the glass slide surface). To isolate tissue, RGB color space was converted to HSV, and a color threshold (threshold = 35) was applied to separate tissue from background. Morphological smoothing was then applied: median filtering to reduce noise while preserving tissue edges, followed by morphological opening (erosion then dilation) to remove small gaps. The resulting tissue regions were tiled into 512x512 pixel patches, with patch coordinates and metadata stored in HDF5 format.
Patch feature extraction with ResNet18: A pre-trained ResNet18 convolutional neural network (ImageNet weights) was used to extract pathological features from each 512x512 patch. Specifically, the first 3 residual blocks of ResNet18 were used, mapping each patch to a 256-dimensional feature vector. This approach leverages transfer learning from natural image recognition to histopathology, capturing low-level textures and higher-order semantic features relevant to tissue structure.
Clinical feature preparation: Clinical variables were selected in collaboration with lymphoma specialists. Continuous variables (such as white blood cell count) were converted to clinically meaningful categories (normal, decreased, increased). Variables with more than one-third missing values were excluded. For remaining variables, categorical missing values were imputed using the mode and continuous missing values using the median. Multi-category variables were processed using One-Hot Encoding to prepare them for machine learning models.
The image-based model used an approach adapted from CLAM (Clustering-constrained Attention Multiple instance learning), a weakly supervised framework designed for whole slide image classification. The core concept of CLAM is that each WSI is treated as a "bag" of many instances (patches), and the model learns to assign attention scores to each instance, then aggregates patch-level features into a single slide-level representation. This eliminates the need for pixel-level or region-level annotations, relying only on slide-level R/R labels.
Attention mechanism in detail: In this study, each patient's WSI is represented as a bag X containing M patches, where each patch is a 256-dimensional vector. Two multilayer perceptron (MLP) layers compress each 256-dimensional patch to a 128-dimensional vector. The attention network uses two weight matrices (Ua and Va, each 64x128) as a shared backbone, plus a final weight parameter (Wa, 1x64). The attention score for each patch is computed using a combination of hyperbolic tangent (tanh) and sigmoid (sigm) activations applied through these weight matrices, then softmax-normalized across all patches in the slide. The slide-level feature H is the weighted sum of all patch features using these attention scores, resulting in a 128-dimensional slide representation fed to the final classification layer.
Random forest for clinical features: For the clinical and molecular pathological model, a random forest (RF) algorithm was used. RF trains an ensemble of decision trees and provides feature importance scores based on the Gini index and out-of-bag error rate. The hyperparameters (n_estimators, max_depth, min_samples_split) were tuned using the Optuna framework over 100 iterations on the validation set, yielding optimal parameters of n_estimators = 80, max_depth = 9, min_samples_split = 6, and a corresponding accuracy of 0.675. Feature importance scores were used to rank all candidate clinical and molecular variables, and the top features were selected for model training. In total, 42 features were retained after this importance-based selection step.
Fusion model via compact bilinear pooling: To combine image features and clinicopathological features, the authors applied compact bilinear pooling (CBP). Clinical features were first embedded through an MLP to produce a vector representation. This clinical embedding and the attention-pooled image feature vector were then fused using CBP, and the fused representation was passed to a final classification layer to produce R/R risk predictions. CBP is a computationally efficient approximation to the outer product of two feature vectors, capturing feature interactions between image and clinical modalities without the dimensionality explosion of full bilinear pooling.
The three models were evaluated on both the validation and test sets, with performance reported as mean AUROC plus or minus standard deviation across 10 repeat trials. The H&E image-only model achieved a mean AUROC of 0.65 +/- 0.12 in the validation dataset and 0.65 +/- 0.08 in the test dataset. The clinical and molecular pathological model achieved a mean AUROC of 0.70 +/- 0.10 in the validation dataset and 0.67 +/- 0.07 in the test dataset. The fusion model combining both modalities achieved a mean AUROC of 0.71 +/- 0.07 in validation and 0.70 +/- 0.04 in testing.
Clinical features dominate feature importance rankings: Among the top nine features identified by random forest importance ranking, rituximab use was a particularly strong predictor (P = 0.0023 in survival analysis). Patients were split into rituximab-treated and non-rituximab-treated subgroups, and six features were analyzed via Kaplan-Meier survival curves in each subgroup. In patients not treated with rituximab, ECOG performance status and disease stage were both significantly associated with prognosis (P less than 0.05). After rituximab treatment, disease stage was no longer significantly correlated with prognosis, indicating that rituximab modified the prognostic significance of staging.
BCL-2 expression and erythrocyte-related variables: BCL-2 positivity (defined as at least 70% of tumor cells expressing BCL-2) was associated with lower survival probability. This association was statistically significant in the rituximab-treated subgroup, consistent with the established biology that BCL-2 overexpression promotes anti-apoptotic resistance to chemotherapy. Erythrocyte count also emerged as a relevant feature: patients with decreased erythrocyte count showed inferior overall survival compared to those with normal counts. Similarly, abnormal hemoglobin was associated with worse outcomes, and higher IPI scores (3 or above) were associated with inferior overall survival regardless of rituximab use.
Fusion model comprehensive metrics: Beyond AUROC, the fusion model demonstrated accuracy of 0.658, precision of 0.640, recall of 0.643, and F1 score of 0.629. The image-only model had accuracy of 0.50, precision of 0.32, recall of 0.38, and F1 score of 0.32. The clinical-only model showed accuracy of 0.61, precision of 0.60, recall of 0.54, and F1 score of 0.57. The fusion model consistently outperformed both unimodal approaches across all metrics.
To make the image-based model interpretable, attention scores were calculated for all patches extracted from correctly classified slides. Scores were normalized from 0 to 1 and mapped to a divergent color scale: red for high-attention patches (contributing most to the model's prediction) and blue for low-attention patches (contributing least). The resulting heatmaps were overlaid on the original H&E images, allowing pathologists to visually inspect which tissue regions drove the model's R/R prediction.
Pathologist review of representative cases: Two pathology experts reviewed representative H&E images and their corresponding attention heatmaps from one non-R/R patient and one R/R patient. In the non-R/R case, the H&E image showed a low density of tumor cells and a rich background with pronounced fibrosis, consistent with a less aggressive tumor microenvironment. In the R/R case, the image revealed a high density of tumor cells and a relatively sparse background, with few T cells, histiocytes, or dendritic cells, and no significant fibrosis. These morphological differences visible to pathologists corresponded meaningfully to patterns the model had learned to attend to.
Cell density quantification via HoverNet: To quantify cell density distributions across the cohort, the authors applied HoverNet, a nuclear segmentation model trained on the PanNuke dataset (covering 19 different cancer tissue types). HoverNet performed instance-level segmentation and classification of five cell types: neoplastic, inflammatory, connective/soft tissue, dead, and non-neoplastic epithelial. No non-neoplastic epithelial cells were predicted in this DLBCL cohort, yielding four relevant categories. Additionally, 15 nuclear morphological features (area, perimeter, shape metrics) were extracted per nucleus and averaged across each WSI to generate a 15-dimensional morphological feature vector per patient.
R/R versus non-R/R cell density differences: Statistical comparison of cell density distributions between groups revealed that inflammatory cell density was lower in the R/R group than in the non-R/R group, though this difference did not reach statistical significance. Connective tissue cell density, dead cell density, and neoplastic cell density were all higher in the R/R group. The elevation in dead cell density was statistically significant (P = 0.0001). These findings align with known tumor biology: elevated neoplastic and connective tissue cell density reflects tumor cellularity and stromal desmoplasia, while reduced inflammatory infiltrate in the R/R group suggests an immunosuppressive tumor microenvironment less capable of mounting anti-tumor responses.
A notable finding that the authors discuss at length is the lack of significant association between cell-of-origin (COO) classification and R/R status in their cohort (P = 0.84). This is consistent with a 2024 study by Bock et al., which found that COO does not show a significant association (P = 0.285) with primary progressive disease in DLBCL. The R/R group in this study included 67 GCB cases and 50 non-GCB cases, with GCB accounting for 57.3% of the R/R group, similar to prior published data showing GCB cases at 60.8% of R/R DLBCLs. COO classification similarly showed no significant correlation with overall survival in this cohort (P = 0.27). These results suggest that COO alone, derived by immunohistochemistry using CD10, BCL-6, and MUM-1 expression patterns (Hans algorithm), is insufficient for predicting R/R risk.
Rituximab as a prognostic modifier: The identification of rituximab use as one of the most important features in the random forest model is clinically meaningful. In patients not receiving rituximab, both disease stage and ECOG score were significant survival predictors. After rituximab, stage lost prognostic significance while ECOG remained significant, indicating that rituximab effectively levels the playing field across disease stages but cannot compensate for poor performance status. This interaction would be difficult to capture with simple Cox proportional hazard models but is naturally accommodated by the nonlinear, interaction-aware random forest framework.
MYC and BCL-2 double expression: In this cohort, double-expression lymphoma (DEL, defined as MYC and BCL-2 co-overexpression) was present in 25.99% of patients. The double-hit lymphoma (DHL) category (MYC/BCL2 chromosomal rearrangement) was rare at 0.96%. While BCL-2 positivity emerged as a significant survival predictor in rituximab-treated patients, the rarity of DHL in this cohort limited analysis of double-hit biology.
Comparison with the Detrait model: The authors compare their approach to a 2024 machine learning study by Detrait et al., which used 130 DLBCL patients to predict primary refractory disease using demographic, clinical, disease characteristics, first-line therapy, and interim PET-CT scan findings. The fusion model in the current study outperformed three of the five Detrait models and was slightly inferior to two of them. The two superior Detrait models incorporated PET-CT scan results after 2 cycles of treatment, which provide an early treatment response signal that is inherently prognostic. This comparison highlights that incorporating interim imaging data adds predictive value beyond baseline features alone.
Image model performance: The image-only model performed at chance level in terms of accuracy (0.50), reflecting the difficulty of predicting R/R status from H&E patches alone in a small cohort with high morphological heterogeneity. The authors attribute this to three factors. First, whole slide images (WSI) rather than region-of-interest (ROI) crops were used for feature extraction, and while prior work has shown WSI achieves the best performance for certain tasks (e.g., lymph node metastasis prediction in thyroid cancer), WSI-based feature extraction in DLBCL may introduce noise from non-diagnostic background areas. Second, the small dataset size (227 patients) creates a challenging signal-to-noise ratio for deep learning, which typically benefits from thousands of training samples. Third, abstract patch features from ResNet18 may lack the interpretability and specificity needed to capture the complex histological drivers of R/R DLBCL, and incorporating explicit cell ratio analysis, spatial relationships between tumor and immune cells, and tissue component quantification might yield higher performance.
Single-center dataset and absence of external validation: All 227 patients came from a single institution over a 4-year window (2015-2018). There is no independent external test set. The TCGA DLBCL cohort was evaluated but lacked sufficient follow-up and R/R status completeness. Single-center datasets carry inherent risks of selection bias, center-specific staining protocols, and scanner characteristics that may not generalize to other institutions. The lack of external validation is the most significant methodological limitation of this study, and performance in a prospective multicenter cohort may differ substantially from these single-center results.
Treatment era and cohort composition: The study cohort was treated from 2015 to 2018, preceding the widespread adoption of polatuzumab vedotin (approved 2019) and CAR-T therapies as earlier-line treatments. Only 45.37% of patients in the cohort received rituximab, which is lower than expected given contemporary standard-of-care R-CHOP. This reflects the study institution's practice patterns during that era and may limit generalizability to current DLBCL management. Additionally, none of the patients received hematopoietic cell transplantation as second-line therapy, further limiting the cohort's representativeness of current clinical practice.
Future directions: The authors outline a clear roadmap for improvement. Data expansion from multiple institutions will be the primary next step, alongside attempting different image preprocessing strategies (including ROI-based analysis focused on tumor-rich areas and analysis of tumor infiltrating lymphocyte patterns). They envision a comprehensive final model integrating clinical information, laboratory results, immunohistochemical indicators, and pathological images together into a single prediction tool that could assist hematologists in developing personalized treatment plans at the time of initial DLBCL diagnosis. The authors also note that AI-based image analysis can extract high-dimensional features invisible to the naked eye, and reduction in interobserver variability makes these features more objective than standard pathological assessment.