Software/Hardware Co-Design Optimization for Sparse Convolutional Neural Networks

Deep convolutional neural network (DNN) has been widely used in image recognition, target detection, and natural language processing. Unfortunately, the previous state-of-the-art convolutional neural network (CNN) increase model accuracy by increasing the number of layers of the network, which leads to the problem of large models and complex calculations. Previous work has demonstrated that network compression can be achieved by weight pruning approaches, and that specialized hardware can be used to speed up reasoning. However, the format of compressed sparse rows (CSR) or compressed sparse columns (CSC) is generally used in model pruning. The approach leads to coding operations before calculation and decoding operations during calculation by processing units on specialized hardware. And the approach degrades the performance of the model.In this paper, we propose a combination of hardware and software to address above bottlenecks. Firstly, we propose a novel software-based structured pruning, called vector-pruning. Secondly, we design a novel dataflow for our pruning method, called Vector-Sparse, our method eliminates complex coding and decoding Then, we design the corresponding hardware architecture for vector-prune. Finally, we evaluate the performance of our pruning approach for AlexNet on Xilinx xcvu19p-fsva3824-2-e. Experiments show that compared with a state-of-the-art convolution network accelerator, we proposed a software/hardware co-design approach that is 1.01x and 1.3x better in performance, effectively improving the inference speed of the model on Field Programmable Gate Array (FPGA).