Joint Task Offloading and Resource Allocation in Multi-UAV Multi-Server Systems: An Attention-based Deep Reinforcement Learning Approach

The multi-access edge computing (MEC) provides opportunities for unmanned aerial vehicles (UAVs) to perform computing-intensive and delay-sensitive applications. To further reduce the time delay and energy consumption of UAVs, in this paper, we study the long-term optimization problem of joint task offloading and resource allocation in a multi-UAV multi-server MEC network (JTORA-MUMS), where the task offloading decision and the allocation of CPU frequency, bandwidth, and transmission power are jointly optimized. Furthermore, we formulate JTORA-MUMS as a Markov Decision Process (MDP) with a discrete-continuous hybrid action space and handle the hybrid action space by mapping part of continuous actions to discrete decisions. To solve JTORA-MUMS, we propose a novel deep reinforcement learning (DRL) approach, DDPG-MHSA, in which a multi-head self-attention (MHSA) based actor-critic model is trained by a deep deterministic policy gradient (DDPG) algorithm. Extensive experiments show that the proposed DDPG-MHSA approach outperforms state-of-the-art DRL-based methods and conventional heuristics, with good generalization to the number of UAVs and computing task properties.