Spectral–Spatial Adversarial Multidomain Synthesis Network for Cross-Scene Hyperspectral Image Classification

Cross-scene hyperspectral image (HSI) classification has received widespread attention due to its practicality. However, domain adaptation (DA)-based cross-scene HSI classification methods are typically tailored for a specific target scene involved in model training and require retraining for new scenes. We instead propose a novel spectral–spatial adversarial multidomain synthetic network (S2AMSnet) that can be trained on a single source domain (SD) and generalized to unseen domains. S2AMSnet improves the robustness of the model to the unseen domain by expanding the diverse distribution of the SD. Specifically, to spatially and spectrally generate diversified generative domain (GD), the spectral–spatial domain-generation network (S2DGN) is designed, and two S2DGNs with the same structure but not shared parameters are enabled to generate diversified GD through two-step min–max strategy. A Multidomain mixing module is employed to expand the diversity of the GD further and enhance their class-domain semantic consistency information. In addition, a multiscale mutual information (MI) regularization network (MMIRN) is used to constrain the S2DGN so that the intrinsic class semantic information of its generated GD does not deviate from the SD. A semantic consistency discriminator with spectral–spatial feature extraction capability is utilized to capture class-domain semantic consistency information from diverse GD to obtain cross-domain invariant knowledge. Comparative analysis with eight state-of-the-art transfer learning methods on three real HSI datasets, along with an ablation study, validates the effectiveness of the proposed S2AMSnet in the cross-scene HSI classification task. The codes of this work will be available at
https://github.com/daxichen/S2AMSnet
.