Clinical Motivation Melanoma is the most lethal form of skin cancer, but early diagnosis yields a 92% five-year survival rate. Accurate automated segmentation of skin lesions in dermoscopy images is a fundamental prerequisite for computer-aided diagnosis (CAD) systems. Without precise lesion delineation, downstream feature extraction and classification steps cannot reliably identify diagnostic features.
Core Challenge Automatic dermoscopy segmentation is difficult due to artifacts (hair, gel bubbles, ruler markers, dark corners), indistinct boundaries, low contrast between lesion and surrounding skin, and the high variability in lesion size, shape, color, and texture. Classical segmentation methods using histogram thresholding or clustering perform poorly on these challenging real-world images.
Resolution Independence Problem Standard deep learning segmentation networks (FCN, U-Net, SegNet) require fixed-size input images, necessitating downsampling that loses fine-grained boundary detail. This reduces segmentation output resolution, which is problematic when lesion borders must be accurately delineated for clinical feature extraction.
Proposed Solution The paper proposes a novel four-step pipeline combining YOLOv3 for lesion localization and GrabCut for high-resolution segmentation. YOLO's detection bounding box provides the initialization needed to automate GrabCut (previously semi-automatic), enabling fully automated high-resolution segmentation independent of input image dimensions.
Hair Artifact Impact Hair is one of the most common and problematic artifacts in dermoscopy images. Hair strands crossing the lesion create false edges that confuse boundary detection algorithms, potentially leading to fragmented segmentation results or boundaries that trace hair rather than the true lesion edge.
Top-Hat Transform for Detection Hair detection uses morphological black top-hat transform, which identifies elongated dark linear structures in the image. The transform produces a binary mask highlighting hair pixels by computing the difference between the image and its morphological closing, effectively isolating thin dark structures like hair.
Inpainting for Removal Detected hair pixels are removed using inpainting - an image reconstruction technique that fills detected artifact regions with interpolated values from surrounding tissue. This produces a hair-free image that better reflects true skin and lesion appearance, providing cleaner input for subsequent detection and segmentation steps.
Preprocessing Quality The effectiveness of hair removal preprocessing directly impacts downstream segmentation quality. By eliminating strong edge signals from hair, the preprocessing allows the YOLO detection and GrabCut segmentation to focus on true lesion boundaries rather than artifact boundaries.
YOLOv3 Lesion Detection YOLOv3 (You Only Look Once version 3) is a real-time object detection network that simultaneously predicts bounding boxes and class probabilities across the entire image in a single forward pass. Training on labeled dermoscopy images enables YOLO to localize skin lesions with tight bounding boxes, providing automatic initialization for GrabCut without manual user input.
GrabCut Segmentation GrabCut is an iterative graph cut algorithm that separates foreground from background using color information. Pixels within the bounding box are modeled as potential foreground (lesion) and pixels outside as definite background. The algorithm alternates between Gaussian Mixture Model fitting and graph cut optimization, progressively refining the segmentation boundary.
Resolution Independence Unlike CNN-based segmentation that processes fixed-size images, the GrabCut algorithm operates on the original full-resolution image. YOLO's bounding box coordinates are simply scaled to the original image dimensions. This means GrabCut generates high-resolution segmentation masks at the native resolution of the dermoscopy image, preserving fine boundary detail.
Morphological Post-Processing The final step applies morphological image processing operations including closing (to fill small holes within the segmented lesion), opening (to remove small spurious foreground regions outside the lesion), and connected component analysis to retain only the largest contiguous region as the final lesion mask.
ISBI 2017 Dataset The primary evaluation dataset is the ISBI 2017 Skin Lesion Analysis Towards Melanoma Detection Challenge dataset, containing 2000 training, 150 validation, and 600 test dermoscopy images of 8-bit RGB format with resolutions ranging from 540x722 to 4499x6748 pixels. Images include melanoma, benign nevi, and seborrheic keratosis labels with expert-annotated segmentation ground truth.
PH2 Dataset The PH2 dataset from Hospital Pedro Hispano, Portugal, contains 200 dermoscopy images (80 atypical nevi, 80 common nevi, 40 melanoma) at standardized 768x560 resolution, all acquired with the same 20x magnification lens. Unlike ISBI 2017, the PH2 dataset's controlled acquisition conditions provide a more uniform evaluation environment.
Evaluation Metrics Segmentation performance was measured using sensitivity (true positive rate for lesion pixels), specificity (true negative rate for background pixels), accuracy, Dice coefficient (harmonic mean of precision and recall over pixels), and Jaccard index (intersection over union). These complementary metrics capture different aspects of segmentation quality including boundary precision and area overlap.
YOLOv3 Training Training images were resized to 512x512 resolution for YOLO training. Each image was labeled with bounding box annotations around the lesion for object detection training. Data augmentation was applied during training to improve detection robustness across the range of lesion sizes, shapes, and positions in clinical dermoscopy images.
ISBI 2017 Results On the ISBI 2017 test set, the proposed pipeline achieved 90% sensitivity, outperforming other deep learning-based methods including U-Net variants and FCN architectures evaluated on the same dataset. The high sensitivity is particularly important clinically, as it indicates reliable detection of true lesion pixels with low false negative rates.
Competitive Performance on Other Metrics While sensitivity was the strongest metric, the pipeline also obtained competitive accuracy, specificity, Dice coefficient, and Jaccard index values compared to other methods in the literature. The Dice coefficient and Jaccard index results demonstrated substantial agreement between automated and expert manual segmentations.
PH2 Results On the more controlled PH2 dataset, the pipeline achieved results close to existing methods. The PH2's standardized acquisition conditions reduce the image variability challenge, making the benefit of the YOLO-GrabCut pipeline less pronounced compared to the more diverse ISBI 2017 dataset.
Resolution Quality Advantage A qualitative advantage of the approach is producing high-resolution segmentation boundaries that preserve fine detail at the native image resolution. This is difficult to quantify with standard metrics but is clinically relevant for lesion border analysis and feature extraction in dermoscopy applications where boundary morphology is diagnostically important.
GrabCut Performance on Fuzzy Boundaries GrabCut relies on color contrast between foreground and background, which can fail on lesions with gradual color transitions or where the lesion color blends with surrounding skin. Future work could investigate learned foreground/background models tailored to dermoscopy to improve GrabCut's handling of low-contrast boundaries.
Multi-Lesion Images The current pipeline assumes a single lesion per image. In clinical practice, dermoscopy images may occasionally contain multiple lesions or partial lesion views. Extending the YOLO detection component to handle multi-object scenarios would improve robustness to these clinical edge cases.
End-to-End Training Potential The two-stage pipeline (YOLO then GrabCut) is not trained end-to-end. Future work could explore differentiable approximations of GrabCut or attention-based segmentation networks guided by detection outputs, enabling joint optimization that could further improve segmentation accuracy.
Clinical Validation Segmentation performance on benchmark datasets does not directly translate to clinical diagnostic utility. Prospective studies evaluating whether AI-assisted segmentation improves dermatologist accuracy, reduces examination time, or improves treatment decisions in real clinical settings are needed to establish clinical value.