Convolutional neural network channel pruning based on attention mechanism

Convolutional neural networks use convolution as a special operation to achieve powerful inference by fitting a large number of parameters to the result. However, the sheer number of parameters does not allow us to fully understand how the network works, and research on network tailoring has been emerging in recent years to understand the exact mechanics of network operation. In this paper, by introducing the attention mechanism SEBlock in the Imagenet 2017 champion network SEnet, we find out the preference for input channels in the attention mechanism and propose a corresponding channel evaluation mechanism based on this preference to achieve the goal of allowing convolutional neural networks to perform on-channel pruning operations on the network while getting the performance improvement brought by the attention mechanism. In the CIFAR-10 task, the VGG16 network was able to outperform the original network by only 0.02% after 20% cut, and the Resnet family of networks was able to outperform the original network even with a 10% cut.