ACENet: Auxiliary Context-Information Enhancement Network for RGB-D Indoor Scene Semantic Segmentation

Although networks with traditional convolutional neural network (CNN) backbones have achieved remarkable results in red-green-blue–depth (RGB-D) semantic segmentation, convolution cannot easily capture remote dependencies. Transformers encounter no such difficulties and can obtain richer global information. Therefore, we design an auxiliary context-information enhancement network (ACENet) using Swin Transformer as an auxiliary branch to assist the CNN backbone network. First, we input the outputs of each stage of Swin Transformer and ResNet-34 into a module that integrates local and global information and strengthens segmentation through multi-scale fusion and external attention. Then, we add spatial-depth information to the RGB image through a multi-modal fusion module to obtain a high-quality segmented image. Comprehensive experiments show that the proposed ACENet achieves state-of-the-art performance on two common indoor-scene RGB-D semantic-segmentation datasets.