Compressing Runtime Memory Usage via Activation Remapping for Deploying Deep Neural Networks on MCUs

Deploying deep neural networks (DNNs) on microcontroller units (MCUs) has received increasing attentions. Most existing DNN compression algorithms focus on reducing the parameters of DNN to fit the storage constraints of MCUs. However, runtime memory of MCUs is more limited, and these methods are insufficiently optimized for memory usage, which lower the inference efficiency of DNN models on MCUs. Therefore, we propose a runtime memory compression method based on activation remapping to optimize the runtime memory usage on MCUs. By analyzing the frequency distribution of activation values, we introduce Huffman encoding and remap activation values by dynamic range merging to compress the runtime memory usage of MCUs. In addition, a global frequency table based on activation distributions is designed to further reduce the computation and storage overheads on MCUs. Experimental results show that our method can improve the memory compression ratio of MobileNet by up to 26.9% with the accuracy loss of less than 1%, compared with three state-of-the-art methods.