Joint Optimization on Trajectory, Computation and Communication Resources in Information Freshness Sensitive MEC System

Unmanned aerial vehicle (UAV) assisted mobile edge computing (MEC) can provide flexible coverage and computation services. It is attractive for emergency search, traffic control, disaster response, as well as other applications with desperate requirements for information freshness. In this work, we investigate a three-layer multi-UAV assisted MEC system with stochastic task arrival. In order to minimize the age of information (AoI), we jointly optimize the computation offloading, UAV trajectory and communication resources. In particular, we propose a heterogeneous federated multi-agent reinforcement learning algorithm based on a heterogeneous actor-critic framework. All entities in the three-layer MEC system act as both the actor to interact with the environment and the critic to learn optimal policies. Since the training of each agent in the framework is independent, a novel federated updating method is further adopted to enable cooperation among entities and solve the problem of non-stationary environment. Extensive simulation results validate that the proposed scheme outperforms the baseline schemes in terms of both the convergence and resulted AoI in the multi-UAV assisted MEC system.