Prostate cancer is one of the most common cancers in men, with an estimated 161,360 new cases and 26,730 deaths in the USA in 2017 alone. Magnetic resonance imaging (MRI) has become a routine tool for prostate examination because of its excellent soft-tissue contrast and ability to depict the gland and surrounding structures in detail.
Before treatment planning, radiation therapy, or image-guided surgery, clinicians must accurately outline the boundaries of the prostate, a process called segmentation. When done by hand, this is time-consuming and produces results that vary between different readers or even the same reader on different days -- a problem known as inter- and intra-reader variability.
Automated segmentation addresses these limitations by producing consistent, fast results. A reliable automated method could streamline clinical workflows, improve treatment planning consistency, and ultimately contribute to better outcomes for patients with prostate cancer.
Early deep learning segmentation methods used a patch-wise approach, extracting small image patches around each pixel and training a convolutional neural network (CNN) to classify each patch's center. While effective, this required extracting and processing enormous numbers of overlapping patches, leading to heavy computational redundancy.
A breakthrough came when researchers proposed the fully convolutional network (FCN), which processes an entire image in one pass and produces a pixel-wise output map. Instead of classifying one patch at a time, the FCN outputs a dense prediction for every pixel simultaneously, making it far more efficient and preserving spatial context throughout the image.
Building on the FCN idea, architectures like U-Net and V-Net were specifically designed for medical image segmentation. U-Net uses a symmetric encoder-decoder structure with skip connections, while V-Net extends this concept to three-dimensional volumetric data. These networks set strong baselines that new prostate segmentation methods must surpass.
The proposed PSNet (Prostate Segmentation Network) is built by fine-tuning a pre-trained FCN model that was originally trained on a large natural-image database called the PASCAL VOC dataset. This process, called transfer learning, leverages general visual features learned from millions of natural images and adapts them to the specific task of prostate segmentation on MRI.
The network contains seven hidden layers. Early layers capture low-level features like edges and junctions, while later layers learn higher-level, task-specific representations. Only the last three layers are fine-tuned, since early feature detectors transfer well across different imaging domains without needing adjustment.
A key design choice is the use of a weighted cross-entropy loss function to address class imbalance. The prostate occupies only a small fraction of an MRI volume compared to the surrounding background tissue. Without compensation, the network tends to label everything as background. By assigning higher loss to foreground prostate pixels, the training is steered toward correct delineation of the gland.
Data augmentation techniques including image translations and horizontal reflections were applied to the training data to artificially expand the dataset and improve the model's robustness and generalization to new cases.
Performance was measured with four standard metrics. The Dice similarity coefficient (DSC) measures the overlap between the automated segmentation and the manual ground truth, expressed as a percentage -- higher is better. The relative volume difference (RVD) captures how much the predicted volume over- or under-estimates the true volume.
The Hausdorff distance (HD) measures the maximum surface boundary error between the automated and manual segmentations, penalizing large localized mistakes. The average surface distance (ASD) provides a more balanced measure by averaging the boundary errors across the entire surface of the gland.
Experiments used three datasets totaling 140 T2-weighted MRI volumes. An in-house dataset of 41 patients from Emory Hospital was combined with two publicly available challenge datasets: ISBI 2013 (60 cases) and PROMISE12 (50 cases), after removing 11 overlapping subjects. Performance was assessed using five-fold cross-validation, ensuring every patient appeared in the test set exactly once.
PSNet achieved a mean Dice similarity coefficient of 85.0% with a standard deviation of 3.8%, outperforming all three comparison methods tested: FCN trained from scratch (78.9%), V-Net (83.2%), and U-Net (83.6%). PSNet had both the highest average DSC and the lowest standard deviation, indicating it is not only more accurate but also more consistent across patients.
On boundary accuracy, PSNet achieved the lowest Hausdorff distance of 9.3 mm, compared to 11.9 mm for FCN, 9.5 mm for V-Net, and 10.1 mm for U-Net. The PSNet average surface distance of 3.0 mm was likewise the best among all compared methods.
The relative volume difference of 4.1% indicates a well-balanced segmentation with minimal tendency to either over-segment or under-segment the prostate. Using the weighted cross-entropy loss outperformed the Dice loss variant, which only achieved a DSC of 82.3%, confirming the importance of the loss function design choice.
Each MRI volume was segmented in approximately 4 seconds on a standard GPU workstation, making the method practical for clinical use without requiring specialized hardware beyond a consumer-grade graphics card.
Training and inference were implemented in Python using the Caffe deep learning framework. The model ran on an Ubuntu workstation with 32 GB of RAM, an Intel Core i7-6700 CPU, and an Nvidia GTX 1070 GPU with 8 GB of video memory -- specifications representative of a high-end consumer workstation rather than specialized research infrastructure.
Total training time was 20 hours with CuDNN acceleration. The learning rate was set to 1 x 10^-9 and training ran for 80,000 iterations. Weights were initialized from a pre-trained natural-image model and then updated during fine-tuning using stochastic gradient descent with a momentum of 0.99 and weight decay of 0.0005.
One of the key computational advantages of the FCN architecture is that the entire MRI slice can be used directly as input during both training and testing, avoiding the overhead of patch extraction and enabling efficient end-to-end processing.
PSNet demonstrates that transfer learning from natural images to medical MRI is effective for prostate segmentation. To the authors' knowledge, this is the first study to fine-tune a fully convolutional network pre-trained on a large natural-image dataset specifically for prostate MRI segmentation, representing a novel application of this technique in the prostate imaging domain.
The authors note that performance could be further improved by incorporating larger training datasets. The method is designed to be general: the same deep learning framework could be extended to segment other organs or lesions and applied to other imaging modalities such as CT scans and ultrasound images.
Future work will explore custom loss function designs tailored to segmentation tasks, as well as newer CNN architectures such as Mask R-CNN and pyramid scene parsing networks, which may further boost accuracy. The broader goal is to make automated, reliable organ segmentation a routine part of cancer diagnosis and treatment planning workflows.