Least-Squares Based Layerwise Pruning Of Convolutional Neural Networks

In this paper, we propose a new layerwise pruning method to reduce the number of computations needed to evaluate convolutional neural networks (CNN) after training. This least-squares (LS) based pruning method improves state-of-the-art pruning methods as it solves both problems, how to select the feature maps to be pruned and how to adapt the remaining parameters in the kernel tensor to compensate the introduced pruning errors, jointly. Therefore, our method utilizes both correlations between the input feature maps and the structure in the kernel tensor. In experiments, we show that high reduction rates with a small performance degradation can be obtained with our pruning method and that our pruning method performs significantly better than low-rank factorization methods.