Convolutional neural network for medical hyperspectral image classification with kernel fusion

Deep learning for medical hyperspectral image (MHSI) classification is a rising research aspect in recent years. Different from the general RGB image, the hyperspectral image (HSI) has many bands obtain richness information. In this work, convolutional neural network (CNN) and Gabor filter are employed to exploit deep features from MHSI. In the designed classification framework, the CNN kernels learn to capture abstract and discriminate features; however, it needs a large number of data. To solve this issue, a kernel fusion approach based on CNN kernel with Gabor kernel is then developed. Gabor kernels are implemented with dot product the CNN kernel in the first layer; according to backpropagation, it is expected to capture more discriminative shallow features to help CNN improve classification accuracy. Experimental results show that the proposed approach can improve the traditional CNN classification performance, especially in small-sample-size situations.