FPGA Based Acceleration of a Custom Deep Neural Network Model Inference

Deep learning has evolved and become a powerful tool for making real-time decisions in diverse applications. With the advance of complex deep learning algorithms, there is a continuous need for hardware accelerators with high performance, low power, and reconfigurability. FPGA-based hardware accelerators are now widely adopted for deep learning applications due to their high performance per watt and programmable architecture. The availability of various FPGA acceleration frameworks from FPGA vendors has now reduced the hurdle in programming FPGA and the development time. This paper deals with the implementation of a custom CNN model inference in Xilinx Zynq Ultrascale+ MPSOC using Xilinx Vitis AI acceleration framework and Deep Learning Processing Unit (DPU) Intellectual Property Core (IP core). It unveils the capabilities of the Vitis AI framework and demonstrates the ease with which neural network models can be implemented on Xilinx FPGA. The performance is quantified and compared with the processor and GPU-based implementations. The latency achieved in this implementation makes it suitable for real-time applications.