Feature High-Boosting for Semantic Segmentation

Semantic segmentation has been actively studied with the great success of deep learning. Various network architectures for this task have been introduced over last few years, and the current state-of-the-art methods have shown significant advances enough to be applied to practical applications. However, previous methods often fail to segment small objects due to the limit on the spatial resolution of the feature map encoded through the deep network architecture. In this paper, we propose a simple yet effective module for semantic segmentation. Its key components include feature high-boosting, which efficiently highlights boundaries of small objects by simple operations, i.e., sum of the original feature and the corresponding residual, in the latent space. This is fairly desirable to guide network parameters to focus on features of small objects. Experimental results on various benchmarks show that the proposed method successfully improves the performance of semantic segmentation regardless of backbone networks. In particular, the proposed method with the transformer-based decoder achieves 56.4 mIoU on the ADE20K dataset.