Sparsity-Aware Intelligent Massive Random Access Control for Massive MIMO Networks: A Reinforcement Learning Based Approach

Massive random access of devices brings great challenge to the management of radio access networks. Most of the time, the access requests in the network is sporadic. Exploiting the bursting nature, sparse active user detection (SAUD) is an efficient enabler towards efficient active user detection. However, the sparsity might be deteriorated in case of high concurrent request periods. To dynamically coordinate the access requests, a reinforcement-learning (RL)-assisted scheme of closed-loop access control utilizing the access class barring (ACB) technique is proposed, where the control policy is determined through continuous interaction between the RL agent and the environment. The proposed RL agent can be deployed at the next generation node base (gNB), supporting rapid switching between heterogeneous vertical applications, such as mMTC and uRLLC services. Moreover, a data-driven scheme of deep-RL-assisted SAUD is proposed to resolve highly complex environments with continuous and high-dimensional state and action spaces, where a replay buffer is applied for automatic large-scale data collection. An Actor-Critic framework is formulated to incorporate the strategy-learning modules into the intelligent control agent. Simulation results show that the proposed schemes can achieve superior performance in both access efficiency and user detection accuracy over the benchmark scheme for different heterogeneous services with massive access requests.