DRL and Game Theory-based Trajectory Optimization and Task Offloading in Multi-UAV-assisted MEC

In UAV-assisted Multi-access Edge Computing (MEC) systems, UAV trajectories and resource pricing directly determine system utility - improper UAV positioning will lead to increased delay and energy costs for users, while inappropriate pricing will result in insufficient task offloading or UAV overload. This paper aims to optimize UAVs' trajectories, pricing strategies and users' offloading decisions, enabling UAVs to move to suitable locations and provide computing offloading services at appropriate prices, thereby reducing delay and energy, and enhancing the utilities of both UAV and users. The user utility specifically consist of throughput, energy consumption, delay penalties, and expenditures on purchasing computing resources. The UAV's utility consists of computing energy consumption, delay penalties, and revenue from selling computing resources as incentives. We formulate the pre-offloading problem to solve users' offloading selection, and transform the problem into a multi-user non-cooperative game using game theory while proving the existence of Nash equilibrium. Then we model the interaction between UAV and users using Stackelberg game model and prove the existence of Stackelberg equilibrium. We use multi-agent deep reinforcement learning (MADRL) and propose DRL and Game Theory-based Trajectory Optimization and Task Offloading (DGTT) algorithm to solve UAVs' trajectories and obtain the Stackelberg equilibrium solution. We sequentially solve for the pricing strategy and offloading decisions, thereby obtaining the optimized utilities of both UAV and users. Finally, we conduct simulation experiments to verify the feasibility of DGTT algorithm, along with comparative experiments that demonstrate our proposed DGTT algorithm's excellent performance in optimizing UAV and user utilities, while reducing system energy consumption and delay.