Adaptive Wireless Power Transfer Beam Scheduling for Non-Static IoT Devices Using Deep Reinforcement Learning

In this article, we study wireless power transfer (WPT) beam scheduling for a system which consists of IoT devices and a power beacon (PB) using switched beamforming. In such a system, the IoT devices have a non-static behavior (e.g., their location and power requests keep changing) in general, which conventional WPT beam scheduling algorithms are not capable of adaptively dealing with. To address the non-static behavior, we propose a procedure of deep neural network (DNN)-based WPT beam scheduling. In the procedure, the power-deficient IoT devices transmit a common pilot signal simultaneously. Then, the PB effectively provides power to them with a DNN-based WPT beam scheduling policy. In the DNN-based policy, an estimation of the non-static behavior from the received pilot signals and an adaptive beam generation considering the estimated non-static behavior are integrated thanks to the powerful representational capability of DNNs. To allow the DNN-based policy to learn the optimal policy, we propose a Deep WPT Beam scheduling policy Gradient (DWBG) algorithm using deep reinforcement learning. Through the simulation, we show that DWBG achieves a close performance to the optimal policy. This demonstrates that our algorithm can be applied for practical WPT IoT systems with non-static IoT devices.