The inference operation optimization of an improved LeNet-5 convolutional neural network and its FPGA hardware implementation

This paper proposes an improved LeNet-5 convolutional neural network (CNN) named MeLy-Net which is more suitable for implementing on the Field Programmable Gate Array (FPGA) by using quantization technique at the low power consumption. The parallel inference strategies of kernel layer and pooling layer operation in the MeLy-Net are discussed in detail. The high level synthesis (HLS) tool is used to form the accelerator Intellectual Property (IP), the PYNQ-Z2 development board with proceeding system (PS) and programming logic (PL) two parts is used to realize the complete deployment of the FPGA-based accelerator, and the ARM core on the PS side controls the MeLy-Net FPGA-based accelerator on the PL side through the AXI bus. The MNIST dataset tests show the 98.8% accuracy is achieved by the MeLy- Net. It can achieve high-speed handwriting digital image recognition of MNIST dataset under the low power consumption of 2.128W, and can finish 1000 images processing within only 1.8ms.