Progressive Kernel Pruning Based on the Information Mapping Sparse Index for CNN Compression

Network pruning can effectively reduce a model's capacity and computational load, thereby making model deployment in mobile devices less difficult than that without pruning. To improve the pruning rate of the model while maintaining the kernel's feature extraction ability, this paper designs a progressive kernel pruning method for CNN model compression based on the proposed information mapping sparsity index. This method first prunes the kernels in the filter and then prunes the kernels in the convolution layer when the model reaches a certain compression ratio. The whole process is called progressive kernel pruning (PKP). For the kernel pruning process, this paper defines the information mapping sparse index (IMSI), which is used to measure the mapping ability of the convolution kernel related to the amount of information transferred by the convolution operation. When pruning the kernels of filters and layers, according to the IMSI, the kernels with the strongest mapping abilities are retained to transfer as much information as possible with the least number of kernels. Progressive kernel pruning can make use of the characteristic that the model is easy to optimize when kernel pruning in the filter, and it avoids having the model easily fall into local optima when kernel pruning in the layer directly. The experimental results on the CIFAR-10/100 and ImageNet datasets show that compared to the existing CNN model compression methods, the IMSI-based progressive kernel pruning method exhibits better compression performance in processing the model compression tasks that are currently popular. In particular, pruning VGG-16 on CIFAR-10 with our model achieves a compression ratio of 80.8x and an acceleration ratio of 14.8×, which are 5.8× and 4.2× higher than the best results at present, respectively, and the classification accuracy decreases by only 0.59% relative to that of the baseline.