Modularized Multi-Stage Binarized Deep-Model Compression and Performance Recovery

Despite the approaching Internet of Things (IoT) era, most smart devices cannot directly execute complicated deep neural networks (DNNs) due to their limited memory sizes and computing power. Even with model compression, compressing and executing a non-binarized DNN model often needs a codebook for weight calculation, which typically requires acceleration from a Graphics Processing Unit (GPU). Parameter binarization is one commonly used method to reduce computing time and space, but it often seriously compromises the accuracy of the binarized DNN (BNN), which tends to adopt single-stage non-modularized design. To address these issues, we propose a modularized multi-stage BNN model compression with flexible combination of parameter and structure compression to minimize model size while maintaining maximum accuracy possible. The compressed BNNs do not need any codebooks so they better fit into embedded platforms without any GPU acceleration. From our evaluations, the size of a BNN model can be reduced to 1/57 size at the cost of only about a 5% decrease in accuracy, which outperforms the state-of-the-art single-stage compression on BNN models. Furthermore, the proposed method can recover the accuracy of a BNN by 5% even after its size has been further compressed to approximately half its size, demonstrating its potentials in leveraging edge intelligence.