Age of Information Minimization in Status Update Systems with Imperfect Feedback Channel

Status Update System (SUS) are monitoring applications of Internet of Things (IoT). They are formed by a sender that monitors a remote process and sends status updates to a receiver over a wireless channel. For successful monitoring, the sender must keep the status updates at the receiver fresh. This freshness is generally measured using the Age of Information (AoI) metric. The aim of the sender is to find a monitoring and transmission strategy that minimizes the AoI. To find the optimal strategy, the sender needs to accurately track the AoI at the receiver, i.e., it needs to perfectly know whether a transmitted status update is correctly received or not. This knowledge can be achieved by using a feedback channel between receiver and sender to send acknowledge (ACK) or negative acknowledge (NACK) messages. However, in real applications, the feedback channel is not perfect, and the transmission of ACK/NACK messages might fail. This means, the monitoring and transmission decisions have to be made under uncertainty about the receiver's AoI. To overcome this challenge, we introduce the concept of a socalled belief distribution and propose a joint monitoring and transmission strategy at the sender based on reinforcement learning. Our approach, termed Belief Learning, exploits the belief distribution to minimize the AoI at the receiver. Through numerical simulations we show that Belief Learning enables the sender to achieve near-optimal performance with respect to the perfect feedback channel case.