Risk-Sensitive Optimization and Learning for Minimizing Age of Information in Point-to-Point Wireless Communications

When using Internet of Things (IoT) networks for monitoring, devices rely on fresh status updates about the monitored process. To measure the freshness of these status updates, the concept of Age of Information (AoI) is used. However, critical applications, e.g., those involving human safety, require not only fresh updates, but also a low risk of experiencing high AoI values. In this work, we introduce the notion of risky states for these high AoI events. We consider a point-to-point wireless communication scenario containing a sender transmitting randomly arriving status updates to a receiver through a wireless channel. The sender decides, when to send a status update and when to wait for a newer one. The sender's goal is to jointly minimize the AoI at the receiver, the required transmission energy and the frequency of visiting risky states. We present two solutions for this problem using optimization and learning, respectively For the optimization approach, we propose a family of threshold-based transmission strategies, which trigger a transmission whenever the difference between the AoI at the sender and at the receiver exceeds a certain threshold. Our proposed learning approach directly includes our notion of risky states into traditional $Q$-learning As a result, it balances the minimization of AoI and the required transmission energy, with the frequency of visiting risky states. Through numerical results, we show that our proposed risk-aware approaches outperform relevant reference schemes. Moreover, and in contrast to value iteration, their computational complexity does not depend on the set of possible AoI values.