AoI-Guaranteed Bandit: Information Gathering Over Unreliable Channels

In many IoT applications, information needs to be gathered from multiple heterogeneous sources to the base station for real-time processing and follow-up actions. Undoubtedly, information freshness, measured by age of information (AoI), is critical in taking responsive actions. Recent studies have taken AoI into the consideration of transmission scheduling over wireless channels. However, existing studies on guaranteeing AoI either assume error-free wireless channels or priorly known link reliability, which is unrealistic. In this article, we tackle the AoI-guaranteed transmission scheduling problem over an unreliable channel with the aim of throughput maximization, which is modelled as an AoI-Guaranteed Multi-Armed Bandit (AG-MAB) problem. Since the problem has not been studied in the literature even for the oracle case with given link reliability, we first propose an optimal stationary randomized sampling (SRS) policy for the oracle case. For the AG-MAB problem with unknown link reliability, we propose learning algorithms that meet the AoI requirements with probability 1 and incur sublinear regret compared to Oracle SRS, which can also detect the unsatisfiability of the AoI constraint and switch to the fallback policy promptly with guaranteed accuracy. Numerical results show that our algorithm outperforms the AoI-constraint-aware baselines on throughput with per-source AoI requirement guaranteed.