Deep Reinforcement Learning-Based Access Class Barring for Energy-Efficient mMTC Random Access in LTE Networks

Long-Term Evolution (LTE) networks are expected to be a key enabler for the massive Machine-Type Communications (mMTC) service in the 5G context. As highly synchronized access attempts from a massive number of Machine-Type Devices (MTDs) may overload the Physical Random Access Channel (PRACH), the Access Class Barring (ACB) scheme has been officially adopted as a control in LTE specifications. The baseline ACB scheme with fixed barring factor and fixed mean barring time has been shown to prevent the PRACH overload issue at the cost of a sharp increase in access delay. In order to improve the baseline ACB's delay performance, several studies have suggested discarding the barring time and dynamically adjusting the barring factor over time. While neglecting the barring time can indeed bring a significant delay improvement, it may also cause an increased level of energy consumption for the MTDs because an MTD may need to continuously listen in order to obtain the updated barring factor without getting to actually transmit. In this paper, we propose to dynamically tune both the barring factor and the mean barring time using a reinforcement learning approach known as the Dueling Deep Q-Network. Computer simulations show that given a certain tolerance level on access delay and energy consumption, our design can achieve a significantly higher level of energy satisfaction while maintaining a comparable level of delay satisfaction compared to schemes that only focus on tuning the barring factor.