Age-of-Information Bandits with Heterogeneous Data Rates

We consider a system with a sensor tracking a time-varying quantity and sending updates to a monitoring station using one of
$K$
different data-rates for each update. The probability of an attempted update is an unknown function of the data-rate of the update. The metric of interest is the Age-of-Information (AoI), defined as the time elapsed since the sensor made the measurement sent in the latest update received by the monitoring station. The algorithmic challenge is to determine which data-rate to use to minimize cumulative AoI over a finite time-horizon. We propose two policies and characterize their performance via analysis and simulations. One of the key takeaways is that taking the current AoI into account while determining which data-rate to use is key for good performance. In addition, we study the trade-off between AoI and throughput for the system considered.