Power Control in Energy Harvesting Multiple Access System With Reinforcement Learning

The Internet of Things (IoT) application has a crucial need for long-term and self-sustainable operations. Energy harvesting (EH) technique has attracted great attention in IoT as it may significantly increase the network lifetime by using renewable energy sources. In this paper, we study a simple IoT system composed of one base station (BS) and multiple EH user equipments (UEs), where the system control is modeled as a Markov decision process without any prior knowledge assumed on the energy dynamics. The central controller, i.e., the BS, is in charge of scheduling a subset of UEs to access the limited orthogonal channels and regulating transmission power for the scheduled UEs. Applying reinforcement learning (RL) methods in this situation is technically challenging since the state and action spaces are continuous. With a long short-term memory (LSTM)-based algorithm to predict the UEs' battery states, we propose an actor-critic deep Q-network (DQN) RL algorithm to simultaneously deal with the access and continuous power control problem, by considering both the sum rate and prediction loss. The experimental results show that the proposed RL algorithm can achieve better performances when compared with the existing benchmarks.