Photoacoustic imaging (PAI) -- also called optoacoustic imaging -- is an emerging technique that exploits a physical phenomenon first observed by Alexander Graham Bell in 1880: when short pulses of laser light are absorbed by biological tissue, the brief heating causes the tissue to expand and emit ultrasound waves. By detecting those sound waves at the surface of the body, a computer can reconstruct a detailed image of what is inside.
The key advantage of PAI over conventional optical imaging is depth and specificity. While light scatters extensively in tissue and cannot form clear images beyond a millimeter or two, the resulting acoustic (sound) waves travel more cleanly and can be detected from several centimeters deep. PAI provides exceptional contrast for blood vessels because hemoglobin is a strong absorber of near-infrared light -- making it particularly valuable for visualizing tumor blood supply, which is a hallmark of cancer growth.
Beyond structural imaging, PAI enables functional and molecular imaging. By illuminating tissue at multiple wavelengths and analyzing how different molecules absorb different colors of light, PAI can measure oxygen saturation in tumor blood vessels, track molecular contrast agents, and potentially identify cancer at the molecular level. This multi-parametric capability makes PAI a uniquely information-rich imaging tool.
PAI systems face fundamental limitations in how they collect data. Practical ultrasound detector arrays have limited angular coverage, limited bandwidth, and sparse spacing -- meaning they capture an incomplete picture of the acoustic signals. Traditional reconstruction algorithms work well only under ideal conditions that are rarely met in practice. The result is image artifacts, blurring, and loss of fine structural detail. Deep learning (DL) has emerged as a powerful approach to compensate for these hardware limitations.
A second challenge is quantitative PAI (QPAI) -- the goal of measuring the actual concentrations of specific molecules (such as hemoglobin or injected contrast agents) from photoacoustic signals. The problem is that light fluence (intensity) in deep tissue is unknown and spatially varying -- it changes as it passes through different tissues with different optical properties, making it impossible to directly relate signal strength to molecular concentration. Solving this non-linear, ill-posed problem requires the kind of complex function approximation that deep neural networks excel at.
A third challenge is image understanding -- having a computer automatically identify, segment, and classify what is shown in PAI images. For cancer applications, this means detecting abnormal blood vessel patterns, classifying tissue as malignant or benign, and tracking changes over time. Each of these tasks benefits from deep learning's ability to recognize complex spatial patterns in imaging data. This review from Tsinghua University systematically covers all three challenge categories.
One of the most clinically relevant applications is using deep neural networks to classify tissue directly from PAI data. As early as 2016, researchers achieved 95% accuracy classifying prostate tissue into malignant, benign, and normal categories using features extracted from photoacoustic microscopy images. More recently, 3D convolutional neural networks applied to thyroid cancer detection achieved an AUC of 0.96 -- demonstrating that leveraging 3D spatial information substantially outperforms 2D analysis.
For image segmentation -- automatically outlining structures of interest -- the U-Net architecture dominates the field. A sparse variant of U-Net performed vascular segmentation on clinical multi-spectral PAI images with 30 times fewer parameters than the standard model while maintaining comparable accuracy, a critical advantage for real-time clinical processing. Other U-Net variants achieved Dice coefficients above 0.96 for organ boundary detection in animal studies.
Deep learning also addresses motion artifacts, which contaminate PAI of living subjects due to breathing and heartbeat. A simple three-layer CNN applied to in vivo rat brain images corrected motion artifacts more effectively than all non-AI comparison methods. The ability to correct such artifacts in real time is important for translating PAI to clinical settings where patient motion cannot be avoided.
Practical PAI detectors have limited bandwidth (detecting only a narrow range of sound frequencies), which causes blurring and loss of image features. A five-layer deep network applied to raw channel data was shown to nearly triple signal quality metrics like contrast-to-noise ratio and signal-to-noise ratio compared to conventional deconvolution methods, while being 1.6 times faster. Modified U-Net variants have been shown to reconstruct images from multi-frequency arrays, outperforming traditional reconstruction on all quality metrics.
Sparse detector arrays -- where sensors are spaced farther apart than ideal, reducing cost but introducing streak artifacts -- are another target for deep learning. U-Net postprocessing of images reconstructed from just 30 sensors achieved artifact removal in only 20 milliseconds total (reconstruction plus processing), compared with 25 seconds for iterative model-based methods. The FD-UNet variant, in which each network layer is connected to every other layer, consistently outperformed standard U-Net for recovering fine structural details.
The most complex challenge is when multiple non-ideal conditions occur simultaneously -- sparse detectors, limited angular view, and non-uniform speed of sound all affecting the same image. Generative adversarial networks (particularly WGAN-GP) have been applied to remove these combined artifacts, with the discriminator network learning to distinguish real tissue structure from reconstruction artifacts. Combined models that take both raw sensor data and a preliminary reconstructed image as input consistently outperform models that use only one of these data sources.
A critical practical challenge for deep learning in PAI is the lack of training data. Unlike natural image datasets with millions of labeled examples, PAI clinical datasets are small. Furthermore, the ground truth for PAI reconstruction (the true initial pressure distribution in tissue) cannot be directly measured -- only estimated. As a result, most deep learning PAI studies rely heavily on simulated training data generated using physics-based computer models.
For simulating acoustic wave propagation, the open-source k-Wave toolbox is the most widely used tool, implementing a pseudo-spectral method that efficiently simulates how sound travels through tissue. For simulating light scattering (needed for quantitative PAI), Monte Carlo simulation is considered the gold standard -- though it is computationally expensive. Multiple existing medical image databases (including CT scans and MRI data) are repurposed as anatomical templates for generating realistic simulated PA images.
Transfer learning is a key strategy for bridging the gap between simulation and real data: models first trained on simulated datasets are subsequently fine-tuned on small collections of real experimental data. Studies consistently show that transfer learning improves performance on experimental data, though the gap between simulated and real performance remains a field-wide challenge. Standardization efforts by the International Photoacoustic Standardisation Consortium aim to create shared benchmark datasets to accelerate progress.
The most clinically immediate cancer applications of deep learning-enhanced PAI are in breast cancer and skin cancer monitoring, where relatively superficial targets allow higher quality imaging. PAI can map the abnormal blood vessel networks (neo-angiogenesis) that form around tumors, and deep learning enables automated detection of these patterns. Blood oxygen saturation measurement within tumors -- a biomarker of malignancy and treatment response -- becomes clinically actionable when automated by deep neural networks.
A particularly promising development is deep learning applied to low-cost, portable PAI systems. High-power pulsed lasers are expensive and bulky; replacing them with low-cost LED light sources reduces image quality but makes PAI accessible in resource-limited settings. U-Net models trained to enhance LED-based PAI images have demonstrated 30% SNR improvements and roughly 8-fold reductions in the number of image averages needed, making LED-PAI a viable tool for portable point-of-care imaging.
Single-channel data acquisition systems represent another miniaturization opportunity: by using an LSTM-based autoencoder to reconstruct PAI images from a single multiplexed detector channel rather than 120 parallel channels, one system achieved 6-fold faster image reconstruction while maintaining acceptable quality. Such innovations could eventually enable handheld PAI devices -- analogous to today's portable ultrasound systems but providing additional molecular information.
Quantitative PAI (QPAI) aims to measure the concentration of specific molecules in tissue from photoacoustic signals. The key target is blood oxygen saturation, which reflects tumor metabolism and treatment response. The challenge is that light fluence (brightness) is spatially unknown inside tissue, making it impossible to directly convert signal intensity to molecular concentration. Deep learning excels here because it can learn the complex, nonlinear mapping between observed PA signals and underlying molecular distributions.
Modified U-Net architectures applied to multi-wavelength PAI data have measured blood oxygen saturation with mean errors as low as 0.51% in numerical phantom studies -- dramatically outperforming traditional linear spectral unmixing methods. One network combining convolutional layers with recurrent processing reduced unmixing error from 62% (linear methods) to 1.43% using only two wavelengths, enabling near-real-time quantitative imaging at 18.4 milliseconds per frame.
A notable technical insight is that trusted-zone segmentation -- using a separate neural network to identify which tissue regions have sufficiently high signal-to-noise for reliable quantification -- substantially improves the overall accuracy of QPAI. Predictions in low-signal regions are inherently unreliable; filtering these out before reporting results prevents the system from producing confidently wrong quantitative maps that could mislead clinical decision-making.