UAV-based Emergency Communications: An Iterative Two-stage Multi-agent Soft Actor-critic Approach for Optimal Association and Dynamic Deployment

This paper investigates future emergency wireless communication systems based on multiple unmanned vehicles cooperative deployment. A terrestrial carrier vehicle with wireless communication and management capabilities are deployed to release multiple unmanned aerial vehicles (UAVs) which will serve as aerial mobile stations (UAV-BSs) to cover a disaster affected area, forming an emergency Internet of Things (IoT) network. Under the proposed system architecture, we formulate a joint optimization challenge considering the UAV-BSs’ dynamic deployment positions and the association policy between user equipments (UEs) and BSs to maximize the throughput and coverage in dynamic scenarios as a time-varying mixed-integer non-convex sequential programming (MINSP) problem. To solve this problem, we first investigate the impact of decision delay caused by physical networking and computing environment on system performance to illustrate the urgent need for efficient algorithms. Then, a two-stage iterative training algorithm called centralized training multi-agent soft actor-critic with branch-and-cut (CT-MASAC-BAC) is proposed for computing globally optimal solutions. Numerical results show that CT-MASAC-BAC outperforms the heuristic algorithms and other benchmark deep reinforcement learning algorithms in terms of system utility. Furthermore, the experimental results show that the proposed algorithm is scalable with an increasing number of deployed UAV-BSs, contributing to potentially increased performance with more serving UAV-BSs.