Research on Dynamic Labels in Network Pruning

Convolutional neural network compression technology plays an extremely important role in model transplantation and deployment, especially in mobile and embedded hardware platforms with small memory and low computing power, compression technology is even more critical. Convolutional neural network channel pruning technology has developed rapidly in recent years, and a number of excellent pruning algorithms have emerged. The channel pruning technology has gradually developed from the earliest static pruning to dynamic pruning, which adopts different pruning schemes for different inputs. However, the current dynamic pruning scheme needs to introduce multiple modules to predict the mask to prune the feature maps, and some schemes also introduce multiple hyperparameters in the loss function to balance the model accuracy and pruning rate, which leads to The model has difficulty converging during training. We propose a dynamic pruning method, each convolution structure configures a simple prediction module, and generating dynamic labels through the input's norm and similarity to guide the prediction module training, which will not bring new parameters to the loss function. We conducted related experiments on multiple models on the Cifar10 datasets. The experiments on ResNet56 show that our scheme is 1.3% higher than the most advanced scheme in terms of compression rate under the premise of the same accuracy.