Image segmentation for dermoscopy

In this blogpost we will talk about the importance of image segmentation for dermoscopy as well as other medical imaging techniques. For skin cancer screening it is of importance to segment the nevus from the skin.

Image segmentation is a fundamental step in many image processing applications. It labels every pixel to a category. In the medical imaging field, image segmentation is often used to determine tumor boundaries. In the field of autonomous driving, image segmentation is also used to distinguish cars, roads, and people. Numerous applications lead to the rapid development of image segmentation techniques. The most intuitive idea is to choose a threshold and use it to distinguish between different objects. In the Otsu-method an algorithm that automatically selects the threshold in the grayscale histogram for image segmentation. If only one threshold is selected, the image will be divided into background and object. If multiple thresholds are selected, the image will be divided into multiple objects. This method assumes that the same objects have similar grayscale values and different objects have different grayscale values. As shown in Figure 1, on the grayscale histogram, the selected threshold should be the valley between two peaks.

Fig.1 Pixels are segmented by thresholds. All the pixels with a grayscale-value lower than T are treated as background, the rest are foreground

The watershed algorithm treats the grayscale histogram of an image as a geographical structure, with the grayscale value of each pixel representing the height of mountains. The threshold is the horizontal plane. The horizontal plane keeps rising, and when the water in two valleys converge, the point of convergence is the watershed, which is also the boundary of the original image segmentation. The region growing method requires selecting seed points first, then calculating the similarity between the around the pixel and the seed, and grouping the pixels with high similarity into one class. This method suffers from image noise and leaves holes inside one object, which could be filled by morphological operations.

k-means clustering is a statistical method. The clustering core is selected by a subtractive clustering algorithm. After that, the surrounding pixels are clustered using a k-means clustering algorithm to get the segments.

Snakes is an active contour model. Given an initial contour on the image, the model iterates based on the initial contour and improves it more accurately. The purpose of the model iteration is to minimize the energy function. The energy function consists of external energy and internal energy. The external capability describes the image information, which is minimized at the edges, and the internal energy describes the contour profile to make it smoother.
Graph Cuts is another energy optimization method. It maps the image as a weighted graph, treats pixels as nodes. The minimum cut is used to obtain the best segmentation of the images.

Another statistical approach is to consider the image as a Markov Random Field (MRF). From the view of statistics, the image is modeled by treating the grayscale of individual pixels in the image as random variables with a certain probability distribution. AlexNet won the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) in 2012. It proved the potential of the Convolutional Neural Network (CNN) and triggered the rapid development of CNN. Many deep learning-based methods were proposed in many fields including image segmentation.

In a future blogpost we will be talking about how it is possible to extract features from segmented images.