Joint Velocity and Spectrum Optimization in Urban Air Transportation System via Multi-Agent Deep Reinforcement Learning

The emerging concepts of Urban Air Mobility (UAM) and Advanced Air Mobility (AAM) open a new paradigm for urban air transportation. One big challenge is that new aerial vehicles (AV) will quickly saturate the already crowded aviation spectrum, which is an essential resource to ensure reliable communications for safe air operations. In this paper, we consider an air transportation system where multiple AVs are operated to transport passengers or cargo from different sources to destinations along their pre-defined paths. During the flight, the minimum communication Quality of Service (QoS) must be achieved at all times to ensure flight safety. Our objective is to minimize the total mission completion time by jointly optimizing the velocities and spectrum allocation for all AVs. We formulate the optimization problem as a multi-stage Markov game where the optimization variables are coupled together. A multi-agent deep reinforcement learning VD3QN algorithm is proposed to enable cooperative learning among AVs. Additionally, we propose a heuristic greedy algorithm (HGA) and an orthogonal multiple access (OMA) solution as baseline solutions. Extensive simulation results show that our learning-based solution outperforms the baseline solutions under different network configurations.