Online Learning of Optimal Proactive Schedule Based on Outdated Knowledge for Energy Harvesting Powered Internet-of-Things

This paper aims to produce an effective online scheduling technique, where a base station (BS) schedules the transmissions of energy harvesting-powered Internet-of-Things (IoT) devices only based on the (differently outdated) in-band reports of the devices on their states. We establish a new primal-dual learning framework, which learns online the optimal proactive schedules to maximize the time-average throughput of all the devices. Batch gradient descent is designed to enable stochastic gradient descent (SGD)-based dual learning to learn the network dynamics from the outdated reports. Replay memory is deployed to allow online convex optimization (OCO)-based primal learning to predict channel conditions and prevent over-fitting. We also decentralize the online learning between the BS and devices, and speed up learning by leveraging the instantaneous knowledge of the devices on their states. We prove that the proposed framework asymptotically converges to the global optimum, and the impact of the outdated knowledge of the BS diminishes. Simulation results confirm that the proposed approach can increasingly outperform state of the art, as the number of devices grows.