BEGSS! Pruning Candidate Selection for ConvNets

Deep learning has been used to get human-level performance on classification on some of the renowned problems like speech and object recognition. It is very difficult to port deep learning algorithms to resource-limited devices because of their computational cost. However, many ideas have been proposed by the researchers to reduce the computational cost and in this work, we addressed the same problem. We propose one of the promising techniques i.e., Pruning. Pruning starts with learning a large-sized network and then removing the least adversarial parameters. Since in feature map pruning, all the outgoing and incoming kernels are removed, which affects the parameters. Therefore, achieving a high pruning ratio is difficult. To achieve a high pruning ratio, we need to select the candidate for pruning intelligently. Therefore, we propose BEGSS! the pruning candidate selection based on Best of random, Entropy, Gray threshold, Sparsity, and Signal strength. We first computed the statistics of each feature map and select the least adversarial based on these statistics for pruning. The experimental results show that increasing pruning ratio results in degradation of network performance. On Cifar-10 dataset, introducing 70% pruning in the network resulted in 50% performance degradation but on the other hand, the inference time is reduced by 60% of actual inference time. To compensate for the performance degradation, we retrained the pruned network. After retraining the network, we found that the accuracy is improved even for a high pruning ratio. The proposed techniques performed better than existing absolute weight sum voting technique.