Distributed DRL-Based Intelligent Over-the-Air Computation in Unmanned Aerial Vehicle Swarm-Assisted Intelligent Transportation System

Unmanned Aerial Vehicle (UAV)-based edge computing has been widely applied in Intelligent Transportation Systems (ITS) owing to its ease of deployment and high mobility. In this paper, we study intelligent Over-the-air Computation (AirComp) in UAV swarm-assisted ITS. To develop a holistic service framework for UAV swarm, we consider the heterogeneity of Internet of Things Devices (IoTDs) and UAVs. We model the 3D deployment of UAVs, service configuration, bandwidth allocation, the control of computing capacity, and transmission power as a joint optimization problem. To tackle this complex problem, we first propose a dual time-scale architecture based on Deep Reinforcement Learning (DRL). This architecture enables UAVs to achieve seamless coverage of IoTDs on larger time scales, while collaborative UAVs dynamically provide services on smaller time scales. Next, we propose an intelligent AirComp algorithm D2IAC based on distributed DRL to obtain the optimal UAV deployment and dynamic service policies on different time scales. The D2IAC algorithm consists of three sub-algorithms, i.e., TD3-Based UAV Deployment (TBUD), UAV Services Configuration (USC), and REINFORCE-Based Dynamic Service (RBDS). Sufficient experimental results show that the proposed algorithm can achieve 3D deployment of UAVs with coverage improvement from 9% to 36% compared to clustering, center layout, and random algorithms. Regarding dynamic services, compared with the DDPG algorithm, greedy, fixed, and random strategies, the service durations of UAV swarm are improved by 32.95% to 93.72% and the resource utilization is improved by 36.19% to 49.61%.