Deep Reinforcement Learning for NPDCCH Period Adjustment in NB-IoT Networks

In narrowband Internet of Things (NB-IoT) networks, a base station can provide multiple coverage enhancement (CE) levels to accommodate devices with diverse channel qualities for achieving massive connections. The base station can determine a narrowband physical downlink control channel (NPDCCH) period (NP) for each CE level. The base station should allocate uplink and downlink radio resources on an NP-basis for each device in a CE level. The NP length of each CE level significantly affects the efficiency of uplink and downlink radio resource allocation. However, the key challenge in determining an NP length for each CE level is that the NP length should be decided before the uplink and downlink resource allocation and cannot be frequently changed. Therefore, this paper studies the NPDCCH period adjustment problem considering uplink and downlink traffic. The objective is to minimize the radio resource consumption while each device can transmit or receive its data. To overcome the above issue, we propose an algorithm based on deep reinforcement learning to solve the target problem. Compared with the two baselines, the simulation results show the efficacy of the proposed algorithm and useful insights into the design of the NPDCCH period adjustment algorithm for NB-IoT networks.