RAPID: A Single Stage Pruning Framework

Conventional network pruning methods require multiple stages to identify and train a single compact pruned model. This approach has a high computational overhead and requires multiple iterations to train multiple pruned models increasing the total computational cost. In this work, we present a single-stage Random Pruning Online Distillation (RAPID) framework to identify and train multiple pruned models at once. We randomly prune the original network at scratch and then leverage distillation at the later training epochs to transfer information from the original network(teacher) to multiple pruned models(students) online. Extensive experiments demonstrate the effectiveness of the proposed RAPID framework on several datasets and architectures when com-pared with other pruning methods.