Digital Twin Assisted Risk-Aware Sleep Mode Management Using Deep Q-Networks

Base stations (BSs) are the most energy-consuming segment of mobile networks. To reduce the energy consumption of BSs, inactive components of BSs, with a certain activation/deactivation time, can sleep. In this study, we model the problem of BS energy saving utilizing multiple sleep modes as a sequential MDP and propose an online
traffic-aware
deep Q-learning approach to maximize the long-term energy saving. However, there is a risk that BS is not sleeping at the right time and incurs delays to the users. To tackle this issue, we propose to use a digital twin model to encapsulate the dynamics underlying the investigated system and estimate the risk of decision-making (RDM) in advance. We define a novel metric to quantify RDM and predict the performance degradation. Mobile operators can compare the RDM with a tolerable threshold to decide on deactivating SMs, re-training, or activating SMs. We trained an agent using real traffic, obtained from an operator's BS in Stockholm. The data-set contains data rate information with very coarse-grained time granularity. Thus, we propose a scheme to generate a new data-set using the real network data-set which 1) has finer-grained time granularity and 2) considers the bursty behavior of traffic data. Simulation results show that using proposed methods, considerable energy saving is obtained at cost of negligible number of delayed users. Moreover, the proposed digital twin model can predict the performance of the DQN proactively in terms of RDM hence preventing the performance degradation in the network in anomalous situations.