HePiLUT: Resource Efficient Heterogeneous Pipelined CNN Accelerator for FPGAs

Field programmable gate arrays (FPGAs) have gained recognition as a suitable platform for implementing Convolutional Neural Network (CNN)-based algorithms due to their favorable attributes, such as energy efficiency and parallel computing capabilities. Nevertheless, deploying CNNs on resource-constrained devices poses challenges due to the substantial computational demands of CNNs and limited on-chip hardware resources. A majority of prior research predominantly relies on the limited block RAM (BRAM) for storing CNN parameters, and costly digital signal processing (DSP) blocks for executing multiply-accumulate (MAC) operations. To address these constraints, this paper introduces HePiLUt,a novel compiler designed to analyze the CNN structure and parameters, and automatically generates a heterogeneous pipelined CNN accelerator architecture by utilizing Look-Up Table (LUT). In addition, a dataflow controller in the pipelining architecture has been developed to foster a resource-efficient and low-latency CNN accelerator for FPGAs. HePiLUT leverages LUTs to store input image data and execute MAC operations instead of DSP blocks. The efficacy of the proposed CNN accelerator compiler has been evaluated on the Xilinx ZCU102 FPGA platform utilizing the Visual Geometry Group (VGG) architecture and CIFAR-10 dataset. The findings underscore HePiLUT's capability, showcasing a substantial reduction compared to state-of-the-art CNN accelerator design. However, these substantial savings come at a trade-off, as there is a 3% decrease in throughput performance. The proposed HePiLUtsignificantly reduces the FPGA resource utilization and automatic compiler solution in CNN implementation, providing considerable support for sustainable and energy-efficient computing systems.