A Fast Precision Tuning Solution for Always-On DNN Accelerators

Due to the nonvolatility nature of resistive RAM (ReRAM), dynamic operations in the arrays contribute to a much larger portion of power in ReRAM-based neural networks than static power. To reduce the dynamic power of
in-situ
operations with neural parameters, precision-tuning is considered a viable approach of approximate computing to tradeoff excessive computation exactness for power and efficiency gains. However, the switching overhead of precision tuning in hardware severely impacts its effectiveness when the systems need to quickly react to the change of environment, user constraint or input quality. This work for the first time investigates the feasibility of agile precision tuning for neural network accelerators to benefit from approximate computing. The proposed computing in memory (CiM) CNN accelerators fully utilize the normally off characteristics of memristor crossbars to achieve instant network precision tuning without worrying about the model reloading penalty. The ReRAM-based accelerator, with the proposed neural parameter mapping policy and the novel mixed-model training method, induces negligible precision-switching latency and power consumption when compared with traditional variable precision accelerators. In evaluation with state-of-the-art workloads, the proposed ReRAM deep learning and neural network architecture saves 58.3%–62.47% area overhead over the baseline design. We also leverage the proposed ReRAM accelerator architecture to build a novel always-on key-word spotting (KWS) system. The KWS design can switch between different precision modes to capture the relevant sound with high accuracy. The experimental results show the precision-adjustable KWS architecture saves considerable operating energy when fed with realistic test-sets of audio data.