Learning-Based Autonomous Scheduling for AoI-Aware Industrial Wireless Networks

Due to the ever-increasing time-sensitive industrial applications, critical-machine type communication (C-MTC) is a promising technique for timely delivery services in industrial wireless networks (IWNs), where vicinal devices can benefit from device-to-device (D2D) communication for low power consumption and latency. For real-time applications, Age of Information (AoI) is an essential metric that represents the freshness of data from the perspective of destinations. Thus, an AoI orchestration agent for link scheduling in D2D-enabled IWNs is needed. Most existing works on AoI deal with this scheduling in a centralized manner, which cannot afford timely packet delivery requirements for numerous D2D devices. Different from the existing works, a learning-based autonomous AoI and power orchestration agent, namely, L-AoI, is proposed for D2D-enabled IWNs in this article, where D2D devices adaptively compete for wireless resources in a distributed manner. As a result, the global channel state information as well as the actions of other D2D devices are unknown. Hence, D2D devices deal with this uncertainty under the guidance of L-AoI so that AoI constraints can be respected. By leveraging from the belief-based Bayesian reinforcement learning, L-AoI learns the scheduling action profile with strategies of other D2D devices considered so that the spectrum sharing coalitions can be intelligently formed. Both theoretical analysis and simulation are provided to validate the performance of L-AoI in terms of AoI stability and violation ratio.