Joint DRL-Based UAV Trajectory Planning and TEG-Based Task Offloading

The Time-Expanded Graph (TEG) has been widely used to model the dynamically changing network topology of the hierarchical Space-Air-Ground Integrated Network (SAGIN) with unmanned aerial vehicles (UAVs) due to the mobility of UAVs. However, this modeling typically assumes known UAV trajectories, which poses challenges for task offloading when the trajectories are unknown. To address this, we propose a novel approach called Advantage Actor-Critic (A2C) and Sliding Window-based Enhanced TEG (seTEG), referred to as A2C-seTEG. This approach jointly plans UAV trajectories and offloads tasks by dividing the entire trajectory period into smaller sliding time windows. Within each window, UAV trajectories are planned using the A2C model of Deep Reinforcement Learning (DRL) for application in our proposed enhanced TEG (eTEG). By feeding back offloading results from the previous sliding window into the trajectory planning process of the subsequent window, we aim to adjust the DRL training process and optimize both immediate and overall planning and offloading outcomes. The A2C model outperforms its Proximal Policy Optimization (PPO) counterpart in terms of stability, convergence speed, and performance, making it a more effective solution for our scenario. Additionally, we explore the effects of various window sizes and stride lengths on performance, highlighting the trade-offs between algorithmic complexity and overall effectiveness.