ICTNet: Image Complexity-Aware Two-Branch Network with Enhanced Decoding for Real-time Segmentation

Striking a balance between speed and accuracy re-mains a significant challenge in real-time semantic segmentation. Existing methods typically employ an encoder-decoder or a multibranch structure to achieve accurate segmentation while maintaining compactness. However, these methods often overlook the different significance of various pixels and suffer from insufficient spatial information during encoding. To address these challenges, we propose an Image Complexity-aware Two-branch Network (ICTNet) with enhanced decoding. Notably, we introduce image complexity into the spatial branch by supervising a partial of its parameters with image complexity maps. The generated complexity-aware spatial features are fused with context features in the intermediate period of the encoder through our Spatial and Context Fusion (SCFusion) module. The rich context information can provide enhanced object context guidance for spatial features and result in accurate spatial details. To fully integrate the twobranch features and recover sufficient spatial detail information, we design an enhanced decoder which integrates a multi-level context module for feature restoration and develop the Image Complexity Prior Guiding (ICPG) module to fuse the two branch features. The ICPG transforms features into attention maps and modulates the summing of spatial and context features. The fused features are then upsampled through PixelShuffle and bilinear interpolation to produce the final segmentation results. Extensive experiments on the Cityscapes, CamVid, and PASCAL VOC2012 datasets demonstrate the effectiveness of our method. The code is available at https://github.com/dalexin/ICTNet.