DeepAir: A Multi-Agent Deep Reinforcement Learning-Based Scheme for an Unknown User Location Problem

Unmanned Aerial Vehicles (UAVs) are a major component in next-generation network architecture proposals, playing a critical role in problems like dynamic capacity enhancement, user coverage, and task offloading. When smart utilization of the UAVs is missing, these proposals may require sophisticated approaches, including the deployment of additional edge servers and orchestration efforts. A typical challenge arises from the dynamic nature of real-world problems in which the required capacity should be provided at particular times when fixed infrastructure proves insufficient. One of those existing dynamic problems is the unknown user locations in an infrastructure-less environment in which users cannot connect to any communication device or computation-providing server, which is essential to task offloading in order to achieve the required quality of service (QoS). Therefore, in this study, we investigate this problem thoroughly and propose a novel deep reinforcement learning (DRL) based scheme, DeepAir. DeepAir uses four main phases including sensing, localization, resource allocation, and multi-access edge computing (MEC) to provide the corresponding QoS requirements for the offloaded tasks without violating the maximum tolerable delay. To this end, we use two types of UAVs including detector UAVs, and serving UAVs. We utilize detector UAVs as DRL agents which ensure the sensing, localization, and resource allocation phases. On the other hand, we utilize serving UAVs to provide MEC features. Our experiments show that DeepAir provides higher task success rates by deploying fewer detector UAVs in different scenarios with different numbers of users and user attraction points compared to benchmark methods. Thus, DeepAir achieves 59.65%, 86.06%, and 86.72% task success rates for 2, 4, and 6 detector UAVs, respectively, by using 12 serving UAVs, while the most successful benchmark method provides 28.62%, 41.39%, and 61.09% task success rates for the same configuration, respectively.