Pulse-Net: Dynamic Compression of Convolutional Neural Networks

Convolutional Neural Networks (CNNs) are used in a range of computer vision tasks, with state-of-the-art CNNs such as AlexNet and VGG16 constructed using a large number of parameter and multiply-add operations (MACs). These tasks require high computational power and high energy requirements to run the CNNs, making them unsuitable for deployment on In-ternet of Things devices. To overcome this issue and facilitate the use of CNNs on these resource-constrained devices, compression technology through pruning research has gained momentum and is an important tool for improving performance during inference. Our work focuses on pruning unwanted filters and nodes in all layers of a network. The network is pruned iteratively during training via a novel approach we call Pulse-Net, and a significant number of filters/nodes are removed while ensuring any loss in accuracy is within a predetermined range. The unpruned network can be extracted from the original structure for the inference stage. This novel method has an easy-to-set parameter to control the trade-off between accuracy and compression. Pulse-Net gives greater compression, while maintaining competitive accuracy loss, than other reported methods like, efficient convnets, ThiNet and Cross-Entropy Pruning. It also has better robustness against adversarial attacks than other compression and pruning techniques.