Reinforcement Learning Based RSS-Threshold Optimization for D2D-Aided HTC/MTC in Dense NOMA Systems

To fulfill the stringent requirements brought by human-type communication (HTC) along with massive machine-type communication (MTC), device-to-device (D2D) and non-orthogonal multiple access (NOMA) techniques will inevitably be incorporated into dense cellular networks to cater massive connectivity and maintain high spectral efficiency. However, such combination may lead to very complex network topologies and bring challenge in resource allocation, interference management and transmission mode selection. Note the received signal strength (RSS) is an important factor for cellular and D2D mode selection, it can affect multi-access mode determination in D2D-aided HTC/MTC dense NOMA systems. Therefore, the RSS threshold of each cell has great impact on system performance and should be carefully tuned. To this end, we formulate the RSS-threshold selection problem as a decentralized partially observable Markov decision process to maximize the performance for downlink and uplink communications. Accordingly, we employ a multi-agent reinforcement learning based scheme wherein each small base station acts as an agent and chooses the optimal RSS threshold to achieve maximum sum rate by interacting with the environment continuously. Extensive simulation results reveal our proposed scheme can improve the system sum rate and coverage by enhancing the connectivity of massive HTC and MTC devices via D2D and NOMA techniques.