An Accurate Nuclei Segmentation Algorithm in Pathological Image Based on Deep Semantic Network

Cell (nuclei) segmentation is the basic and key step of pathological image analysis. However, robust and accurate cell (nuclei) segmentation is a difficult problem due to the enormous variability of staining, cell sizes, morphologies and cell adhesion or overlapping. In this paper, we extend U-Net with atrous depthwise separable convolution (AS-UNet) for cell (nuclei) segmentation. AS-UNet consists of three parts: encoder module, decoder module and atrous convolution module. The encoder module obtains the high-level semantic information of the cell image layer by layer, while the decoder module gradually recovers the spatial information. The atrous convolution module is composed of cascade and parallel atrous convolution operations. It can extract and combine multi-scale features so that the model can get strong perception ability for both small and large cells. At the same time, the atrous convolution can significantly increase the receptive field of the network model without hurting the segmentation performance or increasing the computational cost. During the training period, Log-Dice loss and Focal loss are combined, while Adam optimization method is employed to optimize the network. In order to increase the penalty for the smaller prediction result of the Dice coefficient, which is carried out by logarithmic operation and the negative value is taken as the Log-Dice loss. The above optimization is beneficial for the convergence speed. Additionally, some data augmentation techniques are applied to increase online data, which contribute to improving the robustness of the model. Compared with several state-of-the-art semantic segmentation algorithms, our method achieves the promising performance on two latest released pathological image datasets.