SievingNet: Ternary-Coded Binary Based Approximate Sieving for Neural Network

In approximate computing acceleration of AI neural network, the pruning techniques are usually simple in concept but suffer in efficiency. In this paper, we propose a simple and efficient approximate approach for floating-point multiply-add accumulator during batch forward propagation. During batch forward propagation, the weights of the neural network remain constant and are worthy of approximately converting to ternary-coded binary fixed-point numbers. Then the associated input floating-point numbers are sieved by checking its exponents only. About 20% of inputs are selected for summation. Finally, at least 3 folds of the computation can be sieved out. From practical evaluation, about 70% of real-time can be reduced with an acceptable accuracy loss.