Joint Scheduling and Power Control in AoI-Oriented WP-IoT Networks: An HVF-Based PPO Approach

In many Internet of Things (IoT) applications, the freshness of data from sensor nodes (SNs) is critical for real-time decision-making. However, due to the limited energy of SNs, the system is faced with a trade-off between energy management and data freshness. Wireless powered IoT (WP-IoT) networks, employing wireless power transfer (WPT) technology, offer a sustainable energy supply. Nevertheless, since SNs rely on energy harvested from wireless power stations (WPS), the uncertainty of energy availability further complicates the balance between data freshness and energy management. Therefore, this paper investigates a joint scheduling and power control problem to minimize the average age of information (AoI) in WP-IoT networks with multiple SNs. The considered WP-IoT system consists of a WPS which charges SNs via WPT, and multiple SNs transmit status updates to a base station (BS) using their harvested energy. The average AoI minimization problem for this system can be formulated by jointly optimizing the scheduling and transmission power of SNs, given the AoI evolution over Rician fading channels. To address the non-convexity and the challenge of an explicit AoI expression, we formulate the problem as a Markov decision process (MDP) with hybrid action spaces involving both discrete scheduling decisions and continuous power control. To solve the MDP, we propose a hybrid value function-based proximal policy optimization (HVF-based PPO) approach, which integrates clipped and unclipped value functions to enhance learning stability and exploration. Simulation results demonstrate that the HVF-based PPO approach significantly reduces the average AoI compared to existing baseline approaches.