Optimizing Age of Information in Energy-Constrained IIoT Networks: A Reinforcement Learning Framework

Age of information (AoI) is a critical metric for ensuring data freshness in carbon-intelligent and energy-efficient industrial Internet of Things (IIoT) networks. We consider a user-specific sensing framework operating in a time division duplexing (TDD)-based IIoT network, consisting of energy harvesting (EH) sensors, users, and a cache-enabled edge node. The proposed framework aims to minimize AoI during data transmission to users while addressing the sensors’ energy constraints. For the uplink transmission of data from sensors to edge node, a Quality-of-Service-aware cognitive radio-inspired nonorthogonal multiple access is employed to enhance spectrum efficiency and minimize cache AoI. During the downlink transmission, upon user’s request, the edge node dynamically decides whether to retrieve cached data or request a fresh update from the sensors. This decision-making process is driven by reinforcement learning (RL) using a Q-table-based approach, where the edge node infers sensor battery levels from received updates and prioritizes user requests accordingly. We formulate this problem as a Markov decision process (MDP) and define an optimal policy that strikes a balance between minimizing AoI and adhering to energy constraints. To achieve this, we develop RL-based solutions, including Q-learning and deep Q-networks (DQNs). Extensive simulations demonstrate that our approach achieves up to 90% AoI reduction while significantly improving energy efficiency, making it well-suited for real-time, delay-sensitive applications in modern IIoT networks.