Online Rate Allocation for AoI Minimization in an Energy Constrained D2D Communication

This paper considers the problem of online rate allocation in a Device-to-Device (D2D) communication system where large-size packets are transmitted over multiple time slots. Moreover, the focus is on the scenario of energy-efficient timely update of packets and considers the minimization of the Age of Information (AoI) metric under an average transmit power constraint. The problem is modeled as a Constrained Markov Decision Process (CMDP) where the objective is to minimize the time average AoI cost while restricting the time average transmit power to a specified threshold. The optimization problem is solved by forming the Lagrangian, followed by the primal-dual approach. The primal problem is an unconstrained Markov Decision Process (MDP) for which the well-established Relative Value Iteration Algorithm (RVIA) can be exploited. However, under the assumption of an unknown probability transition kernel, an in-between post-rate allocation state is introduced, and with the aid of stochastic approximation, we propose an online framework for the rate allocation. Finally, the efficacy of the proposed approach is demonstrated by numerical simulations.