MBFQuant: A Multiplier-Bitwidth-Fixed, Mixed-Precision Quantization Method for Mobile CNN-Based Applications

Deploying Convolutional Neural Network (CNN)-based applications to mobile platforms can be challenging due to the conflict between the restricted computing capacity of mobile devices and the heavy computational overhead of running a CNN. Network quantization is a promising way of alleviating this problem. However, network quantization can result in accuracy degradation and this is especially the case with the compact CNN architectures that are designed for mobile applications. This paper presents a novel and efficient mixed-precision quantization pipeline, called MBFQuant. It redefines the design space for mixed-precision quantization by keeping the bitwidth of the multiplier fixed, unlike other existing methods, because we have found that the quantized model can maintain almost the same running efficiency, so long as the sum of the quantization bitwidth of the weight and the input activation of a layer is a constant. To maximize the accuracy of a quantized CNN model, we have developed a Simulated Annealing (SA)-based optimizer that can automatically explore the design space, and rapidly find the optimal bitwidth assignment. Comprehensive evaluations applying ten CNN architectures to four datasets have served to demonstrate that MBFQuant can achieve improvements in accuracy of up to 19.34% for image classification and 1.12% for object detection, with respect to a corresponding uniform bitwidth quantized model.