Fully Neural Network Low-Bit Quantization and Adjustment of Activation Distributions

Quantization Aware Training (QAT) can usually achieve better performance than Post-Training Quantization (PTQ) in low-bit quantization due to its parameter learnability. However, deploying the network after low-bit quantization to dedicated hardware is still challenging. In particular, convolutional neural networks maintain 8-bit quantization for some of their sensitive layers when quantized at low bit. To enable the underlying hardware circuitry to support two different computational bitwidth at the same time during hardware deployment leads to an increase in cost; In addition, some target detection networks using LeakyRelu, etc. as the activation function can have poor accuracy after quantization due to the asymmetric distribution of activation values. In this paper, we propose a static 4-bit activation value quantization method for the sensitive layer to unify the quantization bits and an Parameterizable LeakyReLU(PLReLU) activation function to adjust the distribution of activation values to make them symmetric, which reduce the hardware cost and improve the accuracy of the quantization network. Our method outperforms current low-bit quantization methods by 1.11% to 1.72% on ImageNet dataset and by 0.9% on PASCAL VOC dataset, demonstrating the contribution of the design in this paper.