Deep learning-enabled realistic virtual histology with ultraviolet photoacoustic remote sensing microscopy

Nat Commun 2023 Deep Learning 6 Explanations View Original
Original Paper (PDF)

Unable to display PDF. Download it here or view on PMC.

Plain-English Explanations
Pages 1-2
The Problem of Positive Surgical Margins and Intraoperative Pathology

When surgeons remove a tumor, their goal is to excise all cancer with a clear surrounding margin of healthy tissue. But despite their best efforts, positive surgical margins -- where cancer cells are found at the edge of the removed tissue -- occur in up to 40 percent of cases. This typically requires additional surgery, radiation, or chemotherapy, causing extra trauma and cost for patients.

The gold standard for confirming surgical margins is formalin-fixed, paraffin-embedded (FFPE) H&E histology, which involves embedding tissue in wax, slicing it into 4-5 micrometer sections, and staining with hematoxylin and eosin dye. This process takes at least 24 hours, far too slow to provide feedback while a patient is still on the operating table.

Frozen section analysis (FSA) offers a faster alternative, providing slides in about 20 minutes. However, FSA has well-documented limitations: ice crystal artifacts distort cell morphology, lipid-rich tissues like breast are difficult to freeze and cut, and false negative rates as high as 36 percent have been reported in prostate and breast cancer surgeries. FSA is consequently not routinely performed for most tumor types.

This study from the University of Alberta introduces a completely different approach: virtual histology using ultraviolet photoacoustic remote sensing (UV-PARS) microscopy. The system scans unstained tissue and uses deep learning to generate H&E-like images in real time, without any chemical processing, staining, or tissue destruction.

TL;DR: Positive surgical margins are common and require additional treatment; current intraoperative methods either take too long or produce poor-quality images, motivating a new virtual histology technology.
Pages 2-4
How UV-PARS Creates Images Without Staining the Tissue

The system uses two complementary ultraviolet imaging mechanisms. UV-PARS (ultraviolet photoacoustic remote sensing) fires a pulsed laser at the tissue and detects tiny pressure waves generated when cell nuclei absorb the light energy. This produces a high-contrast image of nuclei -- equivalent to the hematoxylin (blue/purple) channel of conventional H&E staining.

Simultaneously, UV scattering microscopy detects light scattered by the cytoplasm and extracellular matrix (proteins, collagen, connective tissue), providing contrast equivalent to the eosin (pink) channel of H&E staining. Together, these two co-registered channels provide all the structural information pathologists use to diagnose tissue.

A key advantage over other virtual histology methods is that this approach is entirely label-free -- no dyes, stains, or chemical agents are needed. This is important because applying any exogenous dye risks interfering with subsequent immunohistochemistry, special stains, or genetic analysis of the same tissue, which are often needed for definitive diagnosis.

The system achieves 390 nm lateral resolution and 1.6 micrometer axial sectioning, equivalent to 400x digital pathology. Critically, it scans at rates up to 7 minutes per square centimeter, fast enough to evaluate surgical margins while the patient remains in the operating room. The system also enables optical sectioning of thick, unprocessed tissue by imaging at different depths.

TL;DR: UV-PARS captures nuclei through light absorption while UV scattering captures cytoplasm, together producing label-free images equivalent to H&E staining at surgical speed.
Pages 3-4
Using CycleGAN to Transform Raw Physics Images into Realistic H&E

The raw UV-PARS and scattering images, while informative, do not look like the familiar H&E-stained slides that pathologists are trained to interpret. To bridge this gap, the team used a CycleGAN (cycle-consistent generative adversarial network), a deep learning model that learns to transform images from one visual style to another without requiring matched pairs of images for training.

A standard GAN works by having a generator network create synthetic images and a discriminator network try to distinguish them from real ones. CycleGAN adds a crucial constraint: transforming an image from domain A to domain B and back to A should recover the original. This cycle consistency forces the model to learn true structural correspondence rather than just surface-level stylistic changes, enabling training without precisely aligned paired images.

This unpaired training approach is essential because it is nearly impossible to obtain perfectly matched UV-PARS and H&E images of the same tissue -- the process of fixing, embedding, cutting, and staining for H&E inevitably alters tissue morphology. CycleGAN sidesteps this by learning the statistical style of H&E images as a whole, then applying that style to the physics-based UV-PARS inputs.

The practical benefit is also flexibility: the same UV-PARS dataset can be used to train multiple CycleGAN models for different stain types (not just H&E but also Masson's trichrome, PAS, or institutional-specific stain styles), without the complication of chemically de-staining and re-staining the tissue.

TL;DR: A CycleGAN deep learning model transforms raw UV-PARS physics images into realistic H&E-style histology without requiring paired training examples, enabling style adaptation to any stain type.
Pages 5-8
Quantitative Validation: How Closely Does Virtual Match Real?

The team quantitatively compared 1,921 pairs of virtually-stained and true H&E-stained images using three metrics: multi-scale structural similarity (MS-SSIM), peak signal-to-noise ratio (PSNR), and Pearson correlation coefficient (PCC). Median values were 0.76 MS-SSIM, 21.6 dB PSNR, and 0.82 PCC at full 390 nm resolution.

When images were evaluated at a lower effective resolution of 2 micrometers (comparable to many competing technologies), metrics improved substantially: MS-SSIM reached 0.86, PSNR 26.5 dB, and PCC 0.92. This indicates that large-scale tissue architecture and morphology are accurately reproduced, with minor differences concentrated at the finest sub-cellular detail level.

Nuclear morphology was analyzed using CellProfiler software, measuring cross-sectional area, eccentricity, compactness, and nearest-neighbor internuclear distance. Statistical distributions from virtual and true H&E images were closely matched, with nuclear area differences less than 1 square micrometer on average (compared to a median area of 28.6 square micrometers) -- a difference too small to affect diagnostic interpretation.

Importantly, pathologists were able to identify key diagnostic features in prostate tissue including prostatic carcinoma, perineural invasion, benign stroma, benign glands, and blood vessels. They were also able to assign Gleason scores (3+3, 3+4, and 4+3) to prostate virtual histology images, with validation against true H&E counterparts confirming concordance.

TL;DR: Virtual H&E images closely matched true H&E histology in structural similarity, nuclear morphology, and pathologist-interpretable features including Gleason scoring.
Pages 8-9
Diagnostic Accuracy Study: Virtual Histology vs. Frozen Sections

In a blinded reader study, five pathologists each reviewed 24 breast tissue image pairs and 32 prostate tissue image pairs, independently diagnosing each as benign or malignant based solely on virtual histology images. Pathologist consensus on true H&E images served as the gold standard.

For breast tissue, virtual histology achieved a mean sensitivity of 0.96 and specificity of 0.91. Overall accuracy was 0.93 and Cohen's kappa for intra-observer concordance was 0.86, indicating substantial agreement with the gold standard diagnosis. These results exceed published FSA performance metrics for breast cancer (sensitivity 0.78), where the false negative rate is a significant clinical concern.

For prostate tissue, sensitivity was 0.87 and specificity was 0.94, with a positive predictive value of 0.97, negative predictive value of 0.82, and accuracy of 0.90. The Cohen's kappa again indicated substantial agreement. These values meet the American College of Pathologists' functional requirements for both margin assessment (NPV greater than 0.9 for breast) and lesional tissue identification (PPV greater than 0.9 for prostate).

A separate blinded quality assessment study compared virtual histology directly to frozen section images. Pathologists rated hematoxylin detail, eosin detail, and overall stain quality on a scale of 1 to 4. Virtual histology was preferred in all three metrics, with hematoxylin detail and overall stain quality differences reaching statistical significance (p equals 0.03). Frozen section images frequently fell below the acceptable quality threshold while virtual histology consistently exceeded it.

TL;DR: In blinded pathologist studies, virtual histology achieved 96 percent sensitivity for breast and 87 percent sensitivity for prostate cancer, outperforming frozen sections and preferred by pathologists for image quality.
Pages 9-10
A New Paradigm for Intraoperative Cancer Diagnosis

UV-PARS virtual histology represents a fundamentally different approach to surgical pathology: instead of processing tissue after surgery, it generates diagnostic-quality histological images during the operation, in the same time frame as frozen sections but with superior image quality and without tissue-destructive preparation steps.

The label-free nature of the system is a major practical advantage. Because the tissue is not consumed or chemically altered, the same specimen can still be processed for FFPE permanent histology, immunohistochemistry, molecular testing, and genetic analysis after virtual imaging -- preserving all downstream diagnostic options.

The researchers identify several future directions: parallelized scanning for faster throughput to image multiple breadloafed surgical specimens, a cart-based clinical prototype for operating room integration, and expansion to additional contrast channels including collagen structure and metabolic indicators from autofluorescence. All of these can be captured simultaneously from the existing 266 nm laser excitation.

Larger clinical validation studies are still needed to establish diagnostic equivalence or non-inferiority compared to frozen section and permanent FFPE histology across a diverse range of patients and tumor presentations. But the published results represent a compelling proof-of-concept for what may become a transformative intraoperative diagnostic tool in cancer surgery.

TL;DR: UV-PARS virtual histology can scan unprocessed surgical tissue in under 10 minutes with diagnostic accuracy matching gold-standard pathology, pointing toward real-time margin assessment during cancer surgery.
Citation: Open Access, . Available at: PMC10519961.