Fast generation of high throughput customized deep learning accelerators on FPGAs

Accelerating CNNs has been an active area of research. Research on GPU has led to several well-developed open-source tools such as CAFFE and TensorFlow. However, for FPGA accelerators, such design automation tools are not yet available. We propose an automatic code generation tool that synthesizes high throughput accelerators for CNN inferencing targeting broad types of CNNs and FPGAs. The tool takes as input a high level description of the CNN model and the target FPGA device, and generates fully synthesizable Verilog as output. The tool adopts an algorithm-architecture co-design methodology based on frequency domain convolution. Our proposed algorithm called Concatenate and Pad (CaP), together with our efficient design space exploration, ensure design modularity and scalability (in terms of routing complexity and tool execution time). Users can optionally customize various design parameters, such as FFT sizes and hardware resources to be used. The tool optimizes throughput for a user specified hardware. To illustrate the tool, we generate optimized designs for AlexNet, VGG16 and variations of them (AlexNet∗ and VGG16∗). Experimental results show that for inferencing on these models, throughput of 274.5 GOPS, 660.9 GOPS, 283.2 GOPS and 623.0 GOPS is achieved on the Intel HARP (version 0) platform. The throughput of AlexNet and VGG16 designs outperform state-of-the-art FPGA implementations by 1.85x and 3.53x respectively. The tool is delivered as a Python3 package, and is easily portable onto various computing platforms. Experiments on variety of CNNs and target FPGA devices show that the tool runs in less than 20 seconds on a commodity desktop.