ISQ: Intermediate-Value Slip Quantization for Accumulator-Aware Training

The development of lightweight technologies has made deploying convolutional neural networks on edge devices popular. However, the overflow caused by low-bit accumulators significantly degrades the accuracy of the model. Therefore, there is a challenge in balancing accuracy and low-bit accumulators. In this letter, we propose a novel method applying for training low-bit quantized neural network named Intermediate-Value Slip Quantization (ISQ). ISQ is used to constrain weights to decrease the risk of accumulator's overflow. Besides, we also set a criterion for ISQ to be aware of the bit width of the accumulator. In addition, we propose a method to integrate bias and Batch Normalization (BN) into the ISQ. This allows the computations to be shifted from the floating-point domain to the fixed-point. The experiment results demonstrate that ISQ effectively suppresses the overflow. The model accuracy under the 16-bit accumulator can be restored to 73.16% from 13.7% on CIFAR-100 with the same quantization configuration. Through our method, the design space of the hardware can be explored and the target accuracy can be achieved with the lowest hardware resources.