Throughput Optimization in Grant-Free NOMA with Deep Reinforcement Learning

Grant-free non-orthogonal multiple access (GF- NOMA) is a promising paradigm for reducing the access delay and improving the spectrum efficiency. As the signals of multiple users are superimposed in GF-NOMA systems, each user is required to select a user-specific pilot sequence to distinguish its own signal from the signals of other users. Packet collisions in the uplink occur when multiple users select the same pilot sequence. In this paper, we first formulate a pilot sequence selection problem for aggregate throughput maximization in GF-NOMA systems. We then design a deep reinforcement learning (DRL)- based distributed algorithm for each user to select its pilot sequence via learning from the past pilot sequence selections. The proposed algorithm does not rely on information exchange between the users and does not require centralized scheduling by the base station. Packet-level simulations show that, for the considered system parameters, the proposed DRL distributed algorithm can achieve an average aggregate throughput which is within 90% of the optimal value, and has a better performance than both acknowledgement-based and random selection GF-NOMA schemes.