Energy Efficient and Fast CNN Inference by Exploring Weight Approximation and Computational Reuse

The challenges of convolutional neural networks (CNNs) based AI inference on edge devices include computing complexity, large memory requirements, and high power consumption. Researchers have pursued efficient hardware, dataflow optimization, and new algorithms to tackle these obstacles. This article introduced a novel acceleration methodology designed to significantly enhance the processing speed and power of existing CNN architectures. The proposed approach leverages the inherent static nature of pretrained CNN weights through the prism of linear approximation. Different weight approximation options are evaluated, including within and across kernels. By using the same weight index from different kernels for approximation, computation reuse becomes possible, as the same index from different kernels is sampled with the same data from the input feature map. The proposed approach is evaluated through the design and analysis of four CNN architectures, considering hardware resources, power consumption, and latency. The proposed algorithms are implemented on a gem5-based RISCV simulator, demonstrating a speedup of approximately 2× compared to the baseline. In addition, the proposed CNN accelerators are implemented on Xilinx Kintex 7 Field Programmable Gate Array (FPGA), resulting in a 50% reduction in FPGA hardware resources. When benchmarked with AlexNet, VGG16, SqueezeNet, and ResNet, the proposed approach achieves a 50% reduction in multiplications compared to previous works while maintaining accuracy with loss of 0.9% for VGG16 and 3.1% for AlexNet. This work can be used as a framework to evaluate the different tradeoffs between accuracy, latency, power, and cost of different CNN architectures and applications.