Spectral–Spatial Correlation Exploration for Hyperspectral Image Classification via Self-Mutual Attention Network

Recently, deep learning methods have been widely used to extract spectral–spatial features for hyperspectral image (HSI) classification and dramatically boost the performance. However, most of them usually take the original HSI cube as the input, where spectral–spatial information is mixed together. Consequently, they cannot explicitly model the inherent correlation (e.g., complementary relation) between spectral and spatial domains, limiting the classification performance. To alleviate this issue, a spectral–spatial self-mutual attention network (S
³
MANet) is proposed in this letter. It, respectively, extracts spectral and spatial features via the corresponding feature module. Subsequently, a self-mutual attention module is designed to enhance these features. More concretely, it performs feature interaction to emphasize the correlation of spectral and spatial domains via mutual attention while self-attention is applied to each domain for learning long-range dependencies. Finally, we infer two classification results from the enhanced spectral and spatial features, and a weighted summation is further applied to obtain a joint spectral–spatial results. Experimental results on two public HSI datasets validate that the proposed S
³
MANet could achieve more satisfactory performance in comparison with several state-of-the-art methods.