AI-Enabled UAV Communications: Challenges and Future Directions

Recently, unmanned aerial vehicles (UAVs) communications gained significant concentration as a talented technology for future wireless communications using its remarkable advantages and broad applicability. Furthermore, UAV networks’ high complex configurations and designs encourage researchers to leverage relevant artificial intelligence (AI) techniques for better beyond fifth-generation (B5G)/sixth-generation (6G) services. This article summarizes AI-aided UAV solutions designated for forthcoming wireless networks. Besides, we deliver a comprehensive summary of machine learning (ML) approaches, including their applications and valuable contributions towards effective UAV network implementations, particularly advanced ML ones like bandits, federated learning (FL), meta-learning, etc. Finally, detailed UAV communication-related future research scopes and challenges is highlighted.


FIGURE 1. UAV promising applications.
to automate complicated UAV-related tasks and intelligently 97 improve overall system efficiency can significantly increase 98 the overall UAV's network performance. 99 We highlight most UAV-surveyed work and analyze them 100 as shown in Table 1. Although there are related survey 101 work [12], [13], [14], our paper is up to date and handles 102 new related AI topics such as meta-learning and FL. Although 103 the works in [15] and [16] reviewed RL and DL approaches 104 for UAVs, they didn't directly address up-to-date AI methods 105 for UAV applications, plus still some critical applications 106 are missing. However, all of the previous papers did not 107 directly address AI schemes for UAV applications. In [17], 108 offered a complete overview of certain possible AI applica-109 tions in UAV-based networks by dividing it into supervised 110 and unsupervised approaches. Still, it has a restriction in that 111 it gives a broad overview of all techniques and does not 112 focus on individual lines. The work of [18] surveyed UAVs of 113 different designs with different AI techniques, but it did not 114 overview all methods. Furthermore, the work of [19] defined 115 vital topics connected to UAVs and contemporary machine 116 learning methods and presented a list of relevant courses and 117 surveys. It examines flocks from the perspective of several 118 open topics in which ML may be used to address various flock 119 concerns. Motivated by the enormous importance of UAVs 120 in future daily lives and the continuous development of AI 121 schemes, we investigate different AI-assisted UAV commu-122 nication approaches. Furthermore, future related challenges 123 and possible solution scenarios are highlighted. This paper 124 is organized as follows: Section II surveys Era before AI. 125 Section III highlights different ML schemes and terminolo- 126 gies and their proper applications. Section IV goes over the 127 difficulties that ML-based solutions have solved. Section V 128 summarizes the future work, including different challenges. 129 Finally, Section VI concludes the paper.  131 Herein, we summarize different ordinary mathematical opti-132 mization solutions for UAV problems as shown in Table 2.

133
In [20], the authors considered a single-cell scenario with 134 many UAV-based users. To allow multi-UAV communica-135 tions, they investigated two transmission modes called UAV 136 to network and UAV to UAV. To enable UAV-to-X to commu-137 nications, they designed a cooperative UAV sense and send 138 protocol, then defined the subchannel allocation and UAV 139 speed optimization issue for uplink sum-rate maximization. 140 It maximized sum-rate uplink but in high complexity way. a realistic non-linear mode to optimize the ground terminals' 155 minimal throughput. It optimized trajectory planning and 156 resource allocation but with ideal assumptions. In [23], the 157 authors jointly optimized both the UAV trajectory and NOMA 158 precoding in a UAV-aided NOMA network scenario, where 159 the UAV and BS simultaneously assist ground users (GUs). 160 It Optimized the UAV trajectory and maximized the sum rate 161 but with a long process time.  shortest-path problems, ranking, and multitask bandits. shared between these regional data centers. Its goal is to help target learners enhance their performance on 499 target domains by transferring information from several but 500 related source domains. The need for a significant amount 501 of target-domain data to generate target learners can be 502 decreased. Transfer learning has become a prominent and 503 promising field in machine learning due to its wide range of 504 applications. Domain adaptation is modifying one or more 505 source domains to transfer information and improve the target 506 learner's performance. The domain adaptation method, which 507 tries to narrow the gap across disciplines, is frequently used 508 in transfer learning [67].
Here, the consequence of a decision is frequently unknown, 511 and the effects might fluctuate over time. If choice results 512 reflect a usual range of outcomes or signify a shift in the 513 reward environment, they should have a significant impact 514 on behavior and learning. As a result, practical learning and 515 decision-making need the capacity to assess both expected 516 and unexpected uncertainty (connected to the variability of 517 findings) (associated with the variability of the environment). 518 Understanding the computational and neurological basis and 519 impacts of these two forms of luck and the interconnec-520 tions between them is critical for understanding adaptive 521 learning [68].
522 it suggested a UAV trajectory planning model for data col-601 lection intending to minimize expired data packets across the 602 sensor system and then relaxed the cryptic original issue into 603 a min-max-age of information (AoI)-optimal route scheme. 604 It solved the UAV path planning with unknown channel 605 states but with specific area. In [81], the authors studied 606 the topic of providing the optimum quality of service (QoS) 607 in UAV-assisted cellular networks. To effectively optimize 608 the usefulness of the UAV, it has suggested a combination 609 design of access point selection and UAV path planning. It has 610 presented a DRL-based method to teach the UAV to seek 611 places with superior channel states and a game theory-based 612 access point selection algorithm to allow users to select the 613 correct access point autonomously based on the cost function. 614 It minimized the content delivery delay but battery life time 615 remained short. In [82], the authors investigated The The 616 cache-enabling UAV NOMA networks,, which UAV base 617 stations aid, and are designed for a mix of augmented reality 618 and traditional multimedia applications. DRL optimizes user 619 association, NOMA power allocation, UAV deployment, and 620 UAV caching placement altogether to reduce content delivery 621 time. It controlled continuous action space but with single 622 agent. In [83], it proposed a UAV-aided MEC framework, 623 as several UAVs. with varying trajectories fly over the target 624 region and assist the ground based user equipment. By opti-625 mizing each UAV's trajectory and offloading decision from 626 all the user equipment, a multi-agent DRL-based trajectory 627 control algorithm can jointly maximize the fairness among all 628 the user equipment and the fairness of user equipment-load 629 of each UAV, as well as minimize the energy consumption 630 of all the user equipment. It managing the trajectory of each 631 UAV independently but it did not take cooperative decision. 632 In [84], the topic of reducing the normalized weighted sum of In [89], as a MEC framework with a renew-677 able power supply, the researchers devised a UAV-assisted 678 compute offloading technique. The suggested model consid-679 ers energy arrival instability, stochastic computing demands, 680 and a changing channel state. Due to the state's complex-681 ity, UAV-assisted computed offloading for MEC based on 682 DRL was proposed to reduce the overall cost, which is the 683 weighted sum of delay, energy consumption, and bandwidth 684 cost. In [90], the authors provided a space-air-ground inte-685 grated network edge/cloud computing design for offloading 686 computation-intensive applications even considering remote 687 energy and computation restrictions, where flying UAVs pro-688 vide near-user edge computing and satellites provide cloud 689 computing access. In [91], to determine the best solution for 690 energy-harvesting time scheduling in UAV-assisted device 691 To device (D2D) communications, the authors suggested a 692 unique model based on DRL. The UAV is considered to fly 693 around a central point to make the system model more realis-694 tic. The D2D users move in a continuous random walk. The 695 channel state information encountered during each time slot 696 is randomly time-variant.In [92], the authors presented a UAV 697 system that uses wireless energy transfer to collect data from 698 various geographical regions and deliver it to its destination 699 modeled mobility, energy storage, and data storage patterns 700 to account for time-variant system states detected by the UAV 701 and their effects on decision-making. In [93], for the air-702 ground coordinated communications system, the authors sug-703 gested aerial to ground (A2G)-PMADDPG.   In Table 5, we focus on RL-based UAV solutions that solved In [101], the authors proposed Q-learning-based adaptive 769 geographic routing to improve the converging speed and 770 resource utilization of the geographic routing approaches in 771 vehicular ad hoc networks (VANET). Autonomous vehicles 772 (AVs) are deployed to guide the global transmission path 773 and a Q-learning algorithm is exploited to help each node 774 choose the best next hop in a specific area. In [102], the 775 researchers looked at using UAV-assisted edge caching to 776 help terrestrial vehicle networks transmit high bandwidth 777 content files. It created a combination caching and trajectory 778 optimization issue to judge content location, content distribu-779 tion, and UAV trajectory to improve total network throughput. 780 Due to complex constraints, it chose the optimal path scheme 781 but did not consider saving power. In [104], The authors 782 suggested an online RL UAV-assisted wireless caching sys-783 tem that optimizes the UAV trajectory, transmission power, 784 and caching content scheduling all at the same time. It used 785 the notion of request queues in wireless caching networks to 786 define the combined optimization of online UAV trajectory 787 and caching content delivery as an infinite-horizon ergodic 788 to produce a QoS-optimal solution. It achieved online opti-789 mization of UAV trajectory but it did not calculate time con-790 sumption. In [105], for delay-tolerant wireless sensor network 791 (WSN) applications, the authors suggested an autonomous 792 UAV-based data collection system. The goal is to use a self-793 trained UAV as a flying mobile unit to gather data from 794 ground sensor nodes geographically spread over a particular 795 geographical area during a predetermined period.  solved it with a dynamic spectrum access system and MAB.

854
It maximized the total system rate but with ideal setting 855 assumption. In [58], the difficulty of choosing a gateway UAV 856 is solved. The major goal is to maximize the UAV relays'

897
AI utilization for UAV systems has driven to present numer-898 ous development and savvy arrangements for an endless run 899 of problems as shown in Figure 3. This section briefly surveys 900 the major vital open subjects specified already for UAV issues 901 summarized in Figure 4. From our study, it is clear that more 902 than 40% of researchers used DRL because of its common 903 policy and there are a lot of data to be utilized, but we believe 904 that meta and federated learning will give better accurate and 905 faster results if researchers develop it as it is a hot area to 906 go through and find more methods to solve several problems. 907 Drones are used to obtain confidential data, such as weather 908 forecasting, storm tracking, and precision agriculture.