A Framework for Deep Q-Learning Based Hybrid DVFS Algorithms for Real-Time Systems

In real-time systems, energy consumption is one of the most critical challenges. Dynamic voltage and frequency scaling (DVFS) algorithms have been widely applied to balance the trade-off between performance and power consumption. We first review some classic DVFS methods, e.g., LA-EDF and CCEDF, and then a hybrid DVFS algorithm (Soft-LA2) that we proposed recently. Soft-LA2 needs domain knowledge to set the trade-off parameter manually to achieve energy saving compared with classic methods. Motivated by recent deep learning technologies applied in DVFS, in this paper, we propose a new framework for Soft-LA2. Our method automatically optimizes parameters for Soft-LA2, which leads to more power saving compared with random parameter setting in Soft-LA2. Simulation results show that under certain neural network architecture setting, our method can find the best parameters automatically to achieve power saving performance. Furthermore, with test data, our proposed method saves 4.9% and 9.6% power consumption compared with relevant learning method DQL-EES and random setting Soft-LA2, respectively.