AoI-Aware Status Update Control for an Energy Harvesting Source over an Uplink mmWave Channel

In the new generation networks, the freshness of the data plays a prominent role in real-time systems. The novel metric of the age of information (AoI) measures the elapsed time since the generation of the latest received data. This paper considers a real-time scenario where a source node samples and forwards the measurements to a monitoring center over a millimeter-wave (mmWave) channel. The source node is also equipped with a finite rechargeable battery to harvest energy from the environment. We propose a remote monitoring problem that considers the tradeoff between the minimization of long-term average AoI and the energy usage of the source node. We formulate the problem as an MDP model, and as a model-free reinforcement learning approach, we utilize the Q-learning algorithm to obtain the optimal policy that minimizes the long-term average AoI. Our evaluations investigate the convergence property as well as the impact of changing the problem parameters on the average AoI and average energy consumption. Simulation results show that compared to two other baselines (i.e., random and greedy (myopic) policy), the proposed Q-Learning based algorithm is able to keep the data fresh and consumes less energy by considering the possible future system states.