Dynamic Voltage and Frequency Scaling to Improve Energy-Efficiency of Hardware Accelerators

Neural networks (NNs) have been used in a wide variety of artificial intelligence (AI) applications, including speech recognition, image recognition, automatic robotics, and games. State-of-the-art NNs provide high prediction accuracy at the expense of massive computation that involves large model parameters which consume substantial energy. Though sparse NN s have emerged to reduce the computation and storage overhead, existing specialized DNN accelerators cannot maximize the energy savings when exploiting both dynamic and static sparsity, especially for irregular NNs. In this paper, we propose a dynamic voltage and frequency scaling (DVFS) based hardware accelerator that effectively exploits the dynamic and static sparsity of NNs with dynamic voltage/frequency (V/F) scaling and power gating techniques to reduce both static and dynamic power. To explore the efficiency of DVFS implementation at different granularities, we evaluate both coarse-grained and fine-grained DVFS implementation with different design trade-offs. Further, our proposed DVFS model predicts the dynamic computation workloads as well as V/ F pairs to be supplied to processing elements (PEs) in the hardware intelligently through pre-trained weight vectors. The machine learning based prediction algorithm is deployed to improve the DVFS mode selection accuracy. Our simulation results on AlextNet, VGG16, and ResNet50 show that we can achieve an average dynamic energy savings of 59–66 % and an average static power reduction of 69–80 % compared to the baseline.