IMM-Net: Integrated Multi-stream Memory Network for real-time semantic segmentation

Real-time semantic segmentation is a challenging topic in computer vision. The popular BiSeNet utilizes twobranch approach for spatial information and context information independently. However, we find that its additional, coarser downsampling path to encode spatial information may cause structure redundancy. In this paper, an efficient and effective Integrated Multi-stream Memory module is designed and seamlessly integrated into the whole network architecture (termed IMM-Net) to obtain an extensive range of receptive fields and efficiently integrate multi-scale contextual information. Considering spatial location information is essential to preserving object boundaries, a specific Feature Aggregation Guidance Module is proposed to encode both interpixel semantic relationships and long-range location relationships. Comprehensive experiments on Cityscapes and CamVid datasets indicate our IMM-Net outperforms a few state-of-the-art real-time semantic segmentation methods, achieving a good speed-accuracy trade-off.