Plant Species Recognition Based on Hyperspectral Plant Canopy Images with Deep Learning

Plant species recognition is an important topic in forest remote sensing. In the past, people have used laboratory plant images to design different recognition methods. Some applied machine learning algorithms to perform species recognition on the plant RGB images, while others started to use deep learning (DL) approaches, such as convolutional neural network (CNN), to improve recognition performance. However, two issues arise: Firstly, many species with similar appearance are difficult to be correctly classified by CNN under the limited spectral information of RGB imaging. Secondly, the existing CNN-based classification models are designed for particular datasets and thus are not suitable for plant images. To tackle these issues, this paper proposes a revolutionary framework that combines hyperspectral imaging (HSI) and DL technologies to perform plant species classification with a large number of species. We collected canopy images of 100 plant species via a Visible-NIR hyperspectral camera, and built an plant canopy dataset which consists of 3250 training images and 3250 test ones. Furthermore, we designed a lightweight CNN model that utilizes both 3D CNN and 2D CNN modules to implement spectral information fusion and spectral-spatial features extraction, called Hybrid-CNN. Experimental results show that Hybrid-CNN achieved at least 98% in overall accuracy rate and 0.98 in Kappa coefficient, and significantly outperformed the reference classification models.