Land Use Classification Via Multi-Modal Complementary Feature Fusion and Context Information Enhancement For Optical and Sar Images

Land use classification by fusing optical and synthetic aperture radar (SAR) images has become a research hotspot since it can greatly improve segmentation accuracy. However, due to their different object expression patterns, existing multimodal algorithms have problems such as insufficient utilization of complementary features and inability to solve class imbalance. In this paper, we develop a multi-modal semantic segmentation method based on complementary features fusion and context information enhancement. First, we propose a dual-branch network with no shared weights to extract features of optical images and SAR images respectively. The multi-channel parallel convolution (MCPC) blocks in the network can improve the receptive field of feature extraction. Then, we propose a multi-modal complementary feature fusion (MCFF) module. The two modalities are encouraged to exchange complementary information and suppress redundant information. Finally, for advanced semantic features, we design the context information enhancement (CIE) module to capture multi-scale semantic information and increase feature utilization efficiency to a greater extent. The result of comparative experiments with state-of-the-arts proves the effectiveness of the network.