The immune system plays a dual role in cancer. While chronic inflammation can drive tumor development, the immune system is equally capable of recognizing and attacking cancer cells. The outcome of this battle is partly determined by tumor-infiltrating lymphocytes (TILs) -- immune cells that have migrated into the tumor itself -- which are now recognized as important indicators of the host immune response.
Across many cancer types, higher densities of TILs are associated with better clinical outcomes, including longer disease-free and overall survival. Critically, it is not just the quantity of TILs that matters but also their spatial arrangement. Whether lymphocytes are scattered diffusely throughout a tumor or clustered at its edges may carry different biological and prognostic significance. The concept of the Immunoscore, which quantifies TIL densities in specific tumor zones, has been shown to add prognostic value beyond standard staging systems.
With the rise of immunotherapy -- treatments that harness the immune system to fight cancer -- understanding TIL patterns has become clinically urgent. Therapies such as checkpoint inhibitors work precisely by removing brakes on the immune cells already present in the tumor, making the quantity and organization of those cells a key predictor of therapeutic response.
The Cancer Genome Atlas (TCGA) is one of the most comprehensive cancer genomics datasets in existence, containing molecular data from thousands of tumor samples across dozens of cancer types. Each sample is accompanied by a standard pathology slide stained with hematoxylin and eosin (H&E), which colors cell nuclei blue and surrounding tissue pink, allowing pathologists to visualize cell types and tissue architecture.
Despite the existence of over 5,000 digitized whole-slide images (WSIs) in the TCGA archive, these images have been largely used only to confirm diagnoses and basic tumor characteristics. The rich spatial information encoded in each gigapixel image -- including how many lymphocytes are present, where they are located, and how they cluster -- has remained largely unexplored.
Manual quantification of TILs by pathologists, while possible, is time-consuming, subjective, and impossible to scale to thousands of slides. The authors recognized that deep learning could unlock this hidden resource, systematically extracting immune cell information from images at a scale and resolution that no human could achieve alone.
The authors developed a technique called Computational Staining, which uses convolutional neural networks (CNNs) to automatically identify regions of lymphocyte infiltration in H&E-stained slides. Unlike traditional immunohistochemistry stains that use specific antibodies to label immune cells, this approach extracts the same information computationally from standard diagnostic images already in existence.
Two separate CNNs were built: a lymphocyte CNN that classifies small image patches as either lymphocyte-infiltrated or not, and a necrosis CNN that identifies regions of dead tissue. This second network was critical because necrotic cells can visually resemble lymphocytes and create false positives if not filtered out. Each whole-slide image was divided into 50 x 50 micrometer patches and classified individually.
The lymphocyte CNN used a semi-supervised approach: it was first trained using an unsupervised convolutional autoencoder (CAE) to learn general features of cell nuclei without labeled data, then fine-tuned with expert-annotated examples. This two-step process allows the model to leverage the large amount of unlabeled image data available in TCGA while still incorporating expert knowledge through a relatively modest amount of manual annotation.
The training process involved an iterative feedback loop with expert pathologists, who used a custom web tool called the TIL-Map editor to review CNN predictions, adjust thresholds, and correct errors region by region. The CNN was then retrained on the improved labels. This human-in-the-loop design helped refine the model's accuracy across diverse tumor types.
Applying the computational staining pipeline to 5,455 whole-slide images from 13 TCGA cancer types, the team successfully generated TIL maps for 5,202 images from 4,759 individual patients. Only 4.6% of slides failed quality control, mostly due to low image quality or duplicate slides. This represented the first pan-cancer, image-based assessment of TIL spatial patterns at this scale.
The fraction of tissue infiltrated by TILs varied widely across tumor types. Gastric cancer (STAD) had the highest mean TIL fraction at 14.6%, followed by rectal cancer (READ) at 13.0% and lung squamous cell carcinoma (LUSC) at 11.6%. Uveal melanoma (UVM), included as a negative control because of its known low immune activity, showed only 1% TIL fraction -- confirming the method's ability to distinguish low from high immune activity.
Strong differences were also seen within cancer subtypes. EBV-positive gastric cancer was remarkably TIL-rich, with 25% of tissue infiltrated on average. Among breast cancers, the basal subtype showed the highest infiltration, consistent with prior molecular studies. Among endometrial cancers, the POLE-mutated subtype -- which carries an exceptionally high mutation burden -- was also particularly lymphocyte-rich. These patterns suggest that specific molecular alterations in tumor cells shape the immune microenvironment.
Beyond measuring total TIL fraction, the researchers used affinity propagation clustering to characterize how lymphocytes were organized into spatially coherent groups within each tumor. Each TIL map was described by the number of clusters, their average size, their spatial extent, and measures of within-cluster compactness. These statistics captured whether lymphocytes were scattered broadly or aggregated into tight, discrete foci.
Pathologists also visually classified each TIL map into one of five global structural patterns: Brisk diffuse (dense infiltration throughout the tumor), Brisk band-like (immune cells forming organized borders around the tumor periphery), Non-brisk multifocal (loosely scattered cells in 5-30% of the tumor), Non-brisk focal (limited infiltration under 5%), and None. These categories reflect established pathology classification schemes used in clinical practice.
Structural patterns differed significantly between cancer types. Prostate cancer (PRAD) was strongly enriched in the Non-brisk multifocal pattern, meaning it typically displayed modest and loosely scattered immune infiltrates rather than the brisk diffuse infiltration seen in more immunologically active cancers. This pattern, along with relatively high clustering extent scores, suggests prostate cancer has a distinctive immune microenvironment characterized by spatially organized but moderate TIL clustering.
Survival analyses revealed that beyond total TIL fraction, the spatial clustering structure independently predicted patient outcome in several cancer types. In breast cancer, larger cluster extents were associated with worse survival even after adjusting for overall TIL density. In melanoma, having more numerous but smaller clusters was associated with better survival -- the opposite pattern, consistent with the known responsiveness of melanoma to checkpoint inhibitor immunotherapy.
The computational TIL fraction from images was compared with molecular estimates of lymphocyte content derived from DNA methylation arrays and RNA sequencing (using the CIBERSORT algorithm to deconvolve immune cell proportions). The two methods showed moderate but statistically significant correlation, with Spearman values ranging from 0.20 to 0.45 across most cancer types, and no correlation in uveal melanoma as expected for the negative control.
Perfect agreement between the two approaches was not expected, because they measure different things: the molecular estimate counts the ratio of lymphocyte cells to all cells, while the imaging estimate measures the fraction of tissue area showing lymphocyte infiltration. Additional factors such as the physical distance between the frozen tissue used for molecular analysis and the FFPE tissue used for imaging further limit agreement.
Against direct pathologist scoring of 400 image patches, the CNN showed strong performance. Machine-derived scores were clearly distinct across low, medium, and high TIL content bins assigned by three pathologists, with high intra-pathologist consistency (over 80%) and a statistically significant polyserial correlation of 0.36 between machine and human scores. The CNN slightly outperformed VGG16, a widely used benchmark CNN, by 3.1% in AUROC on a held-out test set.
By integrating image-derived TIL structural patterns with TCGA molecular data, the researchers could examine which types of immune cells were enriched in each structural pattern -- even though individual cell types cannot be distinguished in standard H&E staining. Brisk patterns, which reflect more intense immune infiltration, were associated with higher proportions of CD8 cytotoxic T cells (mean 13.2%), while Non-brisk patterns were relatively enriched in CD4 helper T cells.
This finding links a simple visual observation -- how the infiltrate looks -- to the underlying cellular biology of the immune response. CD8 T cells are the primary anti-tumor effector cells, capable of directly killing cancer cells, while CD4 T cells provide coordinating support. The dominance of CD8 cells in brisk patterns may partly explain why heavily infiltrated tumors tend to respond better to immunotherapy.
The study also identified tumor molecular subtypes with distinctive immune patterns across cancer types. For example, immune subtype C2, which tends toward poor outcomes, showed relatively more Brisk patterns, suggesting that even dense lymphocyte infiltration does not guarantee tumor control in these cases. Immune subtype C4, enriched in macrophage-lineage cells, showed more Non-brisk focal patterns, pointing to how the non-lymphocyte immune compartment shapes the overall tissue architecture of immune infiltration.
This study demonstrates that digitized H&E pathology images contain rich, previously untapped information about the tumor immune microenvironment. The computational staining approach allows immune infiltrate characterization at population scale -- across 4,759 patients and 13 cancer types -- something that would be prohibitively expensive and slow by manual methods alone.
The practical implications extend beyond research. Hospital pathology departments worldwide already possess millions of archival H&E slides. If methods like computational staining could be applied to these existing images, clinicians could gain immune profiling information without ordering additional tests. In the context of deciding whether to pursue immunotherapy or order more detailed immune diagnostics, automated TIL assessment could streamline clinical decision-making considerably.
The researchers made all 5,202 TIL maps publicly available, enabling other scientists to integrate this spatial immune data with their own molecular and clinical analyses. This open-science approach maximizes the value of the TCGA archive and sets a template for future studies combining digital pathology, deep learning, and genomic data to understand the biology of cancer and its immune environment.