Joint Design of Training and Hardware Towards Efficient and Accuracy-Scalable Neural Network Inference

The intrinsic error tolerance of neural network (NN) presents opportunities for approximate computing techniques to improve the energy efficiency of NN inference. Conventional approximate computing focuses on exploiting the efficiency-accuracy trade-off in existing pre-trained networks, which can lead to suboptimal solutions. In this paper, we first present AxTrain, a hardware-oriented training framework to facilitate approximate computing for NN inference. Specifically, AxTrain leverages the synergy between two orthogonal methods-one actively searches for a network parameters distribution with high error tolerance, and the other passively learns resilient weights by numerically incorporating the noise distributions of the approximate hardware in the forward pass during the training phase. Then, we incorporate AxTrain framework in an accuracy-scalable NN accelerator designed for high energy efficiency. Experimental results from various data sets with different approximation strategies demonstrate AxTrain’s ability to obtain resilient neural network parameters for approximate computing and to improve system energy efficiency. And with AxTrain-guided NN models our proposed accuracy-scalable NN accelerator could achieve significantly higher energy efficiency with limited accuracy degradation under joint approximation techniques.