Intelligent Energy Efficiency and Service Reliability Optimization for UAV-Aided Terrestrial Networks

Our study investigates the deployment of Unmanned Aerial Vehicles (UAVs) in areas, such as mountainous regions, where installing terrestrial base stations (TBS) is challenging. This approach extends the coverage area of commercial network services, enhances network capacity, and ensures reliable internet service for ground users (GUs) in remote locations. However, a key challenge remains for terrestrial networks operating in licensed frequency bands, which limits the availability of physical resource blocks (PRBs). This constraint highlights the need for PRB sharing, which introduces interference issues. To address this challenge, we design a federated learning (FL) framework that enables all agents, such as GUs, UAVs, and TBS, to collaboratively learn by interacting with the physical network environment for intelligent dynamic spectrum sharing (DSS) to mitigate interference. Additionally, our FL framework optimizes the placement of UAVs for efficient deployment to maximize network throughput. It also allows GUs and UAVs to adjust their transmit power to achieve energy efficiency that addresses their limited battery storage constraints. To accomplish this, we formulate a mixed-integer nonlinear optimization framework with the objective of minimizing energy consumption while meeting service reliability constraints. The proposed FL framework tackles this optimization problem by transforming it into an unconstrained Markov Decision Process (UUMDP) problem. The GUs employ the asynchronous advantage actor-critic (A3C) algorithm to explore the optimal solution for this UUMDP problem that maintains the time complexity for local model computations even in large-scale network deployments. Additionally, the FL framework provides feedback on the knowledge of all learning agents through global model aggregation to improve local models. Simulation results demonstrate that our approach outperforms the multi-agent deep Q-networks (DQN) method in terms of energy efficiency and service reliability.