Using Fermat number transform to accelerate convolutional neural network

Convolutional neural network (CNN), has achieved a significant breakthrough on image recognition and natural language processing. However, CNNs still suffer from the prohibitive computation complexity. Many efforts have been made to reduce the arithmetic complexity of direct convolution. Aiming at lower multiplication complexity, an efficient convolution architecture based on Fermat number transform (FNT) is proposed in this paper. We first present the FNT algorithm and Overlap-and-Add (OaA) method. To cope with convolution computation in CNNs, the FNT is extended to two-dimensional (2D) FNT and corresponding calculation methodology is introduced. The pipelined FNT architecture is also proposed for efficient realization of FNT. Then an overall architecture of CNN accelerator is given based on OaA FNT. Complexity analysis has demonstrated that the proposed OaA FNT convolution method reduces 5.59× and 2.56× of multiplication complexity compared to direct convolution and OaA FFT method. We also conduct evaluation on the FPGA platform Xilinx Virtex-7 XC7VX485t and comparison is made with respect to convolution throughput and resource efficiency (GOP/s/DSP). The proposed architecture achieves 1.41× convolution throughput with 3.05× less DSPs compared to the state-of-the-art design. 4.29× improvement is gained in terms of the resource efficiency.