Multilevel Dual Attention Residual Network for Scene Segmentation

In this paper, we harness the self-attention mechanism and the integration of multi-scale features to efectively capture rich contextual information, aiming to address the challenge of dense classification. We introduce a novel multi-level dual-attention residual network (MDARNet). This network is designed to adaptively combine local and global features, as well as low-level texture features and high-level semantic features. Specifically, we adopt the U-net as the fundamental model infrastructure. Then, we incorporate spatial attention modules and channel attention modules within the ResNet structure. This enables the selective aggregation of features at every position and across every channel. Notably, we manage to preserve the learned spatial attention matrix (PAM) and channel attention matrix (CAM) of the two attention modules with an extremely negligible storage cost. We not only carry out feature fusion via skip connections to capture contextual information comprehensively, but also utilize the attention matrix derived from low-level texture features to guide and supervise the attention calculation of high-level semantic features. As a result, we are able to obtain more precise segmentation outcomes. We conduct simulation experiments on the Cityscapes dataset, PASCAL VOC2012, and PASCAL Context datasets, and achieve state-of-the-art segmentation results.