Multi-resolution Neural Network Compression Based on Variational Bayesian Inference

In this paper, we investigate multi-resolution model compression for deep neural networks (DNNs) from a Bayesian perspective. By considering the DNN models with channel masks and proposing a resolution likelihood as well as a two-layer sparse prior for the channel masks, we formulate the multi-resolution model compression as a Bayesian inference problem. To solve this problem, we propose a partial update block variational Bayesian inference (PUB- VBI) algorithm which can infer an approximate posterior for the intractable true posterior. The variational posterior and the updating rules are carefully designed such that the proposed algorithm has a low complexity. Simulation results demonstrate that our proposed method can outperform the baselines on various neural network models and datasets.