Compressed Sparse Kernel: Optimization of Pruning for Customized CNNs on FPGAs

Convolutional neural networks (CNNs) have become a widely popular method for performing common computer vision tasks, such as image classification and object detection. While their performance is impressive, many applications struggle to make effective use of these algorithms due to their size and computation requirements. This is because many of the real-world applications of this technology are on small, resource-constrained edge devices. One possible solution to this problem comes from the development of customized hardware designed to perform the run-time computations of a CNN. Due to the low power consumption and high flexibility seen in field-programmable gate arrays (FPGAs), these devices are a popular choice for realizing these systems. In this work, we provide a resource and latency efficient implementation of popular CNNs using the weight pruning technique. Specifically, we propose a novel sparse matrix storage format to significantly reduce the amount of memory consumed by the storage of the parameters on the FPGA. We then derive neural network algorithms which use weights stored in this format directly, without the need to consider weights with a value of zero in any computation. Our design shows a latency improvement of 88% while also reducing the hardware by about 5% when compared to the state-of-the-art technique.