MSPMformer: The Fusion of Transformers and Multi-Scale Perception Modules Skin Lesion Segmentation Algorithm

Aiming at the problems of dermatoscopic images such as hair occlusion, boundary-blurring, and high color variability, this paper proposes the fusion of Transformers and multi-scale perception modules skin lesion segmentation algorithm, which is referred to simply as MSPMformer. Firstly, MSPMformer uses Pyramid Visual Transformer (PVTv2) as an encoder for feature extraction of the whole network’s backbone, extracting feature information layer by layer and outputting multi-scale feature maps. Secondly, with its wide perceptual field, the multi-scale perception module (MSPM) is designed to extract the input multi-scale feature information and focus on the local features by inputting-dependent deep convolution to maximize the feature information extraction and solve the problem of large color differences. Finally, for low-dimensional features, the global adaptive fusion module (GAFM) is proposed to generate global adaptive weights to comprehensively fuse the three layers of feature information at low dimensional, where SCConv reduces redundant features and refines local features to suppress the problem of hair occlusion. For high-dimensional features, the local detail perceptron (LDP) is constructed to capture remote dependencies of the high-dimensional feature information by using local detail features, solve the problem of fuzzy boundaries, and optimize the prediction mask. MSPMformer experiment on the ISIC-2018 dataset and its Dice, Jaccard, and Accuracy are 92.69%, 87.60%, and 96.23%, respectively. Therefore, its segmentation performance is better than that of the existing algorithms. The experimental results show that MSPMformer can effectively solve the problems of hair occlusion, boundary-blurring, and high color variability in skin lesion segmentation, which is able to provide some help for dermatoscopic diagnosis. Our source code will be made available at:https://github.com/bingqi789/Fate.git.