Cooperative Deep Reinforcement Learning based Grant-Free NOMA Optimization for mURLLC

Grant-free non-orthogonal multiple access (GF-NOMA) is a potential technique to support massive Ultra-Reliable and Low-Latency Communication (mURLLC) service. However, the dynamic resource configuration in GF-NOMA systems is challenging due to the random traffics and collisions, which are unknown at the base station (BS). Meanwhile, joint consideration of the latency and reliability requirements makes the resource configuration of GF-NOMA more complex. To address this problem, we develop a general learning framework for signature-based GF-NOMA in mURLLC service taking into account the multiple access signature collision, the user (UE) detection, as well as the data decoding procedures for the K-repetition GF-NOMA. The goal of our learning framework is to maximize the long-term average number of successfully served UEs under the latency constraint. We propose a Cooperative Multi-Agent Deep Neural Network based Q-learning (CMA-DQN) approach to optimize the configuration of both the repetition values and the contention-transmission unit (CTU) numbers. Our results show the superior performance of CMA-DQN over the LE-URC in heavy traffic and demonstrate its capability in dynamically configuring in long term for mURLLC service.