Age-Based Scheduling for Monitoring and Control Applications in Mobile Edge Computing Systems

With the development of Mobile Edge Computing (MEC) and Internet of Things (IoT) technology, various real-time monitoring and control applications are deployed to benefit people’s daily life. The performance of these applications relies heavily on the timeliness of collected environmental information, which can be effectively quantified by the recently introduced metric named age of information (AoI). Although extensive researches have been conducted to optimize AoI under various circumstances, these works commonly require a priori information about the system dynamics that is usually unknown in realistic situations. To design a more practical scheduling algorithm, in this paper, we formulate the AoI minimization problem as a Constrained Markov Decision Process (CMDP) which can be solved by Reinforcement Learning (RL) algorithms without prior knowledge. To improve the running efficiency, we (1) introduce post-decision states (PDSs) to exploit the partial knowledge of the system’s dynamics, (2) perform a batch update in every learning step, (3) decompose the system-level value function into multiple device-level value functions, and (4) propose a heuristic algorithm to find the greedy action. Numerical results demonstrate that our algorithm is highly efficient and outperforms the benchmarks under various scenarios.