Latency-Aware Pruning of Neural Networks via Structure Search with Integer Programming

Latency-aware pruning, which explores the filter structure across each layer by reflecting latency characteristics specific to the hardware, is an effective network compression method for accelerating neural network models in hardware environment. However, due to the high cost of exploring numerous filter structures, previous studies had the drawback of being time-consuming in searching for the optimal filter structure. To address this issue, we propose a latency-aware pruning method that can find the optimal filter structure through Integer Linear Programming(ILP). In this approach, we design an objective function for finding the optimal filter structure by the linear combination of filter importance per layer, and use ILP to quickly find the filter structure that satisfies the target latency. Experimental results show that our method not only improves the performance of pruned networks in terms of accuracy and latency, but also explores filter structures more quickly compared to previous method.