Rounding Shift Channel Post- Training Quantization using Layer Search

Quantization inference has been ubiquitously utilized in edge computing devices, enabling numerous neural networks' efficient deployment. However, the existing methods generally concentrate on classical network structures like VGG-16 and ResNet, ignoring the emerging compact networks such as MobileNets. Specifically, these networks with lightweight architectures have fewer parameters; thereby, less redundancy can be exploited in the quantization stage, which often incurs intolerable accuracy degradations. To handle these problems, we propose a novel Rounding Shift Channel (RSC) post-training quantization method using layer search. We alleviate the accuracy degradation via channel-wise scaling by minimizing a layer-wise reconstruction quantization error. Furthermore, we design a rounding-shift operation to correct biased inference, which has not been explored in previous integer-only-inference studies. The experiments carried on ILSVRC-2012 show that we can achieve 70.1%and 69.9% top-l accuracy for MobileNet V1 and V2, respectively, manifesting the effectiveness of our method. We open-sourced our code at https://github.com/xumengmeng-97IMV-Quantization.