An SCMA-Based Decoupled Distributed Q-Learning Random Access Scheme for Machine-Type Communication

With the dramatic increase in smart machine-type communications (MTC) devices, sparse code multiple access (SCMA) based random access (RA) can be considered as a promising technique for MTC devices to have efficient access into the network. However, the codebook collision problem due to RA may severely decrease the system throughput. In this letter, an optimization problem is formulated to find out optimal strategies in selecting subframes and codebooks for MTC devices, thus maximizing the throughput. To solve this problem, we propose a decoupled distributed Q-learning (DQL) based scheme, in which each device utilizes respectively two Q-learning mechanisms to automatically select its subframe and codebook according to the received feedback signals. By decreasing the dimension of the action set with the decoupled relationship between subframes and codebooks, the proposed scheme could efficiently reduce the convergence time. Numerical results show that the proposed scheme can greatly improve the performance compared with the existing works.