QoS-Ensured Model Optimization for AIoT: A Multi-Scale Reinforcement Learning Approach

Optimizing deep neural network (DNN) models to meet Quality of Service (QoS) requirements in terms of accuracy and computation is of crucial importance for realizing efficient on-device inference in resource-constrained Artificial Intelligence of Things (AIoT). However, most existing works can hardly satisfy the aforementioned QoS requirements since the intrinsic multi-scale characteristic of DNN structures has been seldom considered. In this paper, we formulate a QoS-ensured DNN model structure optimization problem as a novel multi-scale Markov decision process (MSMDP), which can collaboratively decide the DNN structures from different scales. To efficiently solve the above problem, we propose a multi-scale reinforcement learning (MSRL) algorithm, which jointly optimizes block and channel number by interactive multi-scale decision, while ensuring QoS by QoS-based decision evaluation and policy update. Extensive experiments are conducted in both the actual AIoT scenarios and public datasets for different tasks by using different AIoT devices. The results confirm that our proposed MSRL outperforms the baseline schemes in terms of QoS satisfaction, convergence performance, and complexity. Specifically, our algorithm respectively reduces 98.6% computation and 95.7% model size at most while ensuring the QoS compared with the state-of-the-art methods.