Transformer and CNN Hybrid Network for Super-Resolution Semantic Segmentation of Remote Sensing Imagery

Super-resolution semantic segmentation (SRSS) based on Convolutional neural network (CNN) cannot establish long-range dependencies due to limited receptive field, which limits the SRSS to obtain accurate high-resolution (HR) segmentation results from the low-resolution (LR) input images. In this paper, we design a Transformer and CNN hybrid SRSS network that consists of two branches: Transformer and CNN hybrid SRSS branch and super-resolution guided branch. In the Transformer and CNN hybrid SRSS branch, Transformer extracts global context information from the feature map of the CNN, while skip connection is used to retain the local context information extracted from the CNN and combines both features to further improve the segmentation performance. In addition, the super-resolution guided branch is designed to supplement rich structure information and guide the semantic segmentation (SS). We test the proposed method on the ISPRS Vaihingen benchmark data set, and our network is superior to other state-of-the-art methods.