Deep Multiscale Spectral-Spatial Feature Fusion for Hyperspectral Images Classification

Learning representative and discriminative feature that make full use of spectral-spatial information is of cardinal significance for hyperspectral imagery (HSI) interpretation. In this paper, we present a novel unsupervised feature extraction method, deep multiscale spectral-spatial feature fusion (DMS
³
F
²
), for hyperspectral images classification (HSIC). First, pretrained filter banks in VGG16 are transferred to extract deep multiscale spatial structural information about HSI. Second, we propose a new unsupervised cooperative sparse autoencoder method to fuse together the deep spatial feature and the raw spectral information. It achieves both dimensionality reduction and feature fusion, and cooperates on different features to fulfil self-supervised projective learning, which is beneficial in detecting important structure and retaining significant amount of information about the input. In the end, fused spectral-spatial features from multiple scales in VGG16 are further weighted fused and upsampled to obtain the final discriminative feature. Qualitative classification experiments by support vector machines are reported to show the effectiveness of the proposed algorithm. Compared with existing feature extraction techniques, the proposed method is much more effective for HSIC, especially for datasets with less training samples and scenes with complex semantic information and great heterogeneity in terms of geometrical shape.