Semantic Segmentation Based on Spatial Pyramid Pooling and Multilayer Feature Fusion

In recent years, significant progress has been made in semantic segmentation methods. Traditional semantic segmentation methods based on convolutional neural network (CNN) are prone to lose spatial information in the feature extraction stage, and pay less attention to global context information, especially, in some lightweight real-time semantic segmentation networks. This is a huge challenge for semantic segmentation tasks. In addition, although some methods have improved this problem to a certain extent, they are often embedded in specific networks and cannot be applied to other network models. Aiming at these problems, a semantic segmentation method based on multilayer feature fusion is proposed. The flexible and lightweight squeeze–excitation module is used to improve the spatial pyramid pooling (SPP) network, and the accuracy of the semantic segmentation method is further improved by extracting network feature information at different levels. To verify the efficiency and commonality of our methodology, we selected ERFNet and Deeplabv3 networks to experiment on Cityscapes and COCO data sets. Experiments show that our best method can improve 3.1% mIoU and 3.2% mAcc on the Cityscapes data set relative to ERFNet, and at the same time, our method can achieve 61.93 FPS on 1024 $\times $ 512 resolution images and the best improvement of 0.9% mIoU 1.4% mAcc was achieved on the Deeplabv3 network. The experimental results show that the improved multilayer feature fusion structure can improve the accuracy of the semantic segmentation network.