Model-based Weight Quantization for Convolutional Neural Network Compression

Nowadays convolutional neural networks have many applications, from autonomous driving to the medical field. The need for fast deployment of the algorithms, reduced latency, while maintaining a reduced cost created a new paradigm - edge computing - to bring the computational power closer to the user. One limitation for the implementation of convolutional neural networks on edge devices is the large number of parameters, implying high memory usage. As the memory constraints are very strict on an edge device, the need to reduce the memory usage of convolutional neural networks arises. One way is to perform network weights quantization. We propose an adaptive model-based quantization method, with a parameterizable number of quantization intervals. Using the double exponential probability density function, the quantization intervals are determined by taking into account the histogram of trained network weights. For a hardware implementation, we modified a classical convolutional network architecture by replacing fully connected layers with convolutional layers, and changing activation functions. We performed a comparison of the performance of several convolutional network architectures, while the proposed quantization method was applied over the trained network weights. We present experimental results in terms of both accuracy and network size, considering the unquantized network and the quantized network using the proposed method and a non-adaptive one. Proposed method obtained a reduction of the model size to a quarter, in some cases without losing accuracy.