A Spatial-Frequency Domain Joint Mechanism Network for Cross-modal Semantic Segmentation

RGB-Thermal (RGB-T) semantic segmentation has emerged as a promising approach. However, the existing methods ignore the problem of frequency inconsistencies formed by the differences between different modalities. Additionally, traditional global feature construction methods incur significant computational costs to ensure effectiveness. To address these challenges, we propose a spatial-frequency domain joint mechanism network. The Frequency Dilated Hybrid Attention module analyzes the frequency band differences in different modalities, while capturing complex details and high-level semantic information. The Cross-frequency Domain Attention module leverages the frequency domain to estimate scaled dot-product attention, enabling efficient construction of global features. We conducted extensive experiments on the MFNET and PST900 datasets, demonstrating state-of-the-art performance.