Multi-Branch Hybrid Network Based on Adaptive Selection of Spatial-Spectral Kernel for Hyperspectral Image Classification

The current limited sample set and mixed spatial-spectral information make effective feature extraction in hyperspectral image (HSI) classification challenging. To better extract spatial-spectral features, enhance the robustness of the learned features against the orientation and scale changes and improve the convergence of the network used for HSI classification, we propose a multi-branch hybrid network (MHNet) based on adaptive selection of spatial-spectral kernels in this paper. Specifically, we use the Gabor convolutional layer as the first layer of this network model. Since the predefined multi-scale and multi-directional Gabor filters in this layer can better characterize the internal spatial-spectral structure of HSI data from different perspectives, the robustness of the model to orientation-scale changes is enhanced. Then the performance of joint spatial-spectral feature extraction is improved by learning adaptive selective 3D convolution kernels. Subsequently, a two-branch network is employed to further fully extract spatial and spectral information for classification accuracy. Experimental results on three public hyperspectral datasets show that the proposed MHNet not only has better classification performance than several existing widely used machine learning and deep learning-based methods, but also it has fast model convergence.