Joint Deployment and Resource Allocation for Multi-AeBS Networks: A Two-Timescale Optimization Framework Using MADRL

As an important component of the space-air-ground integrated network, aerial base station (AeBS) systems have gained significant attention for their flexibility in mobility and cost-effective construction. Nevertheless, the scarce spectrum resources and difficulty in accessing global information bring necessity and challenges to the deployment and resource allocation of AeBSs. In this paper, we propose a practical two-timescale framework to solve the resource allocation and deployment optimization problem in multi-AeBS networks. Specifically, the subcarrier allocation problem is first transformed into a many-to-one matching game coupled with power allocation and solved in a small timescale. Then, in a large timescale, the AeBS deployment subproblem is transformed into a distributed partially observable Markov decision process (Dec-POMDP), and then a novel multi-agent hypergraph convolutional deep reinforcement learning (MAHGCDRL) is proposed to solve this problem. The proposed MAHGCDRL extracts features of neighboring AeBSs through hypergraph convolutional networks, enabling AeBS agents to achieve better coordination in a distributed manner. Simulation results show that our proposed approach can attain a higher sum rate, and the proposed MAHGCDRL algorithm achieves better learning performance compared to the existing benchmarks in the literature.