Configurable 2D–3D CNNs Accelerator for FPGA-Based Hyperspectral Imagery Classification

Convolutional neural network (CNN) is used efficiently for the classification of hyperspectral imagery (HSI). Both 3-D CNN and hybrid 2D–3D CNN have better performance due to the full extraction of joint spatial-spectral features. Most of previous acceleration research on field-programmable gate array (FPGA) has concentrated on 2-D CNN, few of which were made for 3-D CNN acceleration. More importantly, it is notable that the CNN models for HSI classification have some unique characteristics of computation and memory, different from those for object detection. Thus, the conventional accelerating approaches may be not applicable. To address this problem, we propose a dynamic configurable accelerator architecture suitable for both 2-D and 3-D CNN, ensuring fast development for various networks. The multiple nested loops are optimized delicately and the convolutional layers are fully parameterized, making it easy to scale up the network. Furthermore, we develop the parallelism-oriented memory pattern and data access strategy to optimize the data path and the local buffer. Finally, we implement the proposed architecture for 2-D, 3-D, and hybrid CNNs, validating its effectiveness and reconfigurability. To demonstrate its extensibility, we also prototype the accelerator on two FPGA platforms. We achieve the average and maximum throughput of 18.2/166.18 giga operations per second (GOPS) for HybridSN. To compare with previous FPGA accelerators, a typical baseline 3-D model is also implemented with the average and maximum throughput 160.25/225.87 GOPS, and the performance efficiency achieves up to 1.23 with 10.7% power efficiency gain.