Python to FPGA: Automatic Implementation of custom CNN Model on FPGA

With the rapid development of artificial intelligence neural network technology, the architecture of convolutional neural networks (CNNs) has been evolving towards greater complexity and deeper hierarchical structures. Compared with CPUs and GPUs, Field-Programmable Gate Array (FPGA) are widely used in the inference process of CNNs due to its stability and high efficiency. However, implementing the inference process of CNNs on FPGAs requires great effort and expertise. Therefore, we propose a fast CNN conversion tool that can automatically generate FPGA hardware for customized CNNs inference tasks. The conversion tool takes the Python algorithms of the forward path of the CNN as input and outputs the corresponding FPGA CNN hardware. The Python program used is based on the CNN program based on the Pytorch and TensorFlow frameworks. The tool extracts the key information of each layer in the input CNN model and stores the model structure information in the configuration file. A general-purpose RTL template library is designed for each layer of the CNNs, and by parsing the configuration file, we call different layer templates to generate the complete FPGA CNN hardware. In addition, the weight and bias data of the CNN model trained on the PC will also be automatically converted and stored in the configuration file. We validated the proposed method by automatically converting the LeNet- 5 model implemented in the Pytorch framework and achieved a through-put of 0.25 GOPS and a power-efficiency ratio of 1.08 GOPS/W on the Xilinx FPGA ZYNQ 7010.