Age of Information-Based Optimal Scheduling With Energy Cost Trade-Off for Smart Warehouse: A Deep Reinforcement Learning-Based Approach

Recent advances in the integration of high-speed mobile networks and real-time IoT devices have facilitated in building of smart warehouses, where a set of beacons and Internet of Things (IoT) devices (or source nodes) can monitor the status of various physical processes in a time-critical way. In real-time status monitoring systems, like smart warehouses, quantifying the freshness of the Internet of Things (IoT) data based on the age of information (AoI) metrics becomes quite crucial. As source nodes are battery-constrained, a balanced trade-off between AoI minimization and preservation of source node battery energy is essential. In this paper, in a smart warehouse scenario, we address the problem of minimization of the average weighted sum of AoI subject to the available energy constraint of each source node. We focus on joint optimization of packet transmission and energy may not be suitable for finding a solution in a reasonable time. To address the issue, we develop a deep Q-network (DQN) algorithm that enables us to a simultaneous reduction of the state-space dimension with learning of the age-optimal policy in unknown environmental dynamics. Taking account into the limited energy constraints of the user nodes, the proposed DQN algorithm aims to minimize the long-term average weighted sum-AoI of the user-transmitted data. Finally, we produce multifaceted simulation results in terms of convergence, learning process, and change of average sum-AoI under different energy constraints that shows the effectiveness of the proposed DQN algorithm.