Learning a Deep Structural Subspace Across Hyperspectral Scenes With Cross-Domain VAE

Hyperspectral image (HSI) classification is a small-sample-size problem due to the expensive cost of labeling. As a novel approach to this problem, cross-scene HSI classification has become a hot research topic in recent years. In cross-scene HSI classification, the scene containing enough labeled samples (called source scene) is used to benefit the classification in another scene containing a small number of training samples (called target scene). Transfer learning is a typical solution for cross-scene classification. However, many transfer learning algorithms assume an identical feature space for source and target scenes, which violates the fact that source and target scenes often lie in different feature spaces with various dimensions due to different HSI sensors. Aiming at the different feature spaces between the two scenes, we propose an end-to-end heterogeneous deep transfer learning algorithm, namely, cross-domain variational autoencoder (CDVAE). This algorithm is mainly composed of two key parts: 1) the features of the two scenes are embedded into the shared feature subspace through the two-stream variational autoencoder (VAE) to ensure that the output feature dimensions of the two scenes are identical and 2) graph regularization is used to establish the manifold constraints between source and target scenes in the shared subspace, so as to align the feature spaces. Experiments on two different cross-scene HSI datasets have proved the superior performance of the proposed CDVAE algorithm.