Localization in Unprecedentedly Crowded Airspace for UAVs and SUAVs

The unprecedented proliferation of Unmanned Aerial Vehicles (UAVs), and Swarm Unmanned Aerial Vehicles (SUAVs) has garnered considerable attention from industry and academia owing to their extensive landscape of applications from disaster relief towards smart agriculture. However, flying several UAVs at once poses many challenges to safely and efficiently localize and monitor them. Further, they need to maintain their formation distance to avoid collision between team members and any environmental obstacles. Besides, SUAVs are mainly equipped with an on-board Global Positioning System (GPS) receivers to obtain their positions, but they are not accurate enough and suffer from several vulnerabilities that restrict their applications. Thus, in GPS-denied situations, the acquisition of the positions of UAVs can be assisted by alternative technologies and solutions. This paper is one of the foremost in-depth work that presents, the topic of localization of SUAVs from various perspectives including current research challenges on positioning systems, telecommunication, path planning, along with future opportunities on automated delivery services such as medicine, a remote inspection of industrial sites, and precision agriculture.

UAVs across three main aspects: localization and mapping, 10 obstacle avoidance, and trajectory planning. Furthermore, 11 the authors identified the main challenges to be faced and 12 presented future research directions. Chung et al.    [26] 6 spotted the light on the role of AI methods to improve the 7 performance of UAV networks. It covered the UAV swarm 8 formation methods and presented a detailed overview of 9 AI-enabled routing protocols, as well as the tools, public 10 datasets, and remote experimentation infrastructure used to 11 test these routing protocols. The authors in [27] review the 12 use of UAVs in the sugarcane industry monitoring and man-13 agement and discuss advantages and drawbacks of their use. 14 Meanwhile, the paper [28] is restricted in presenting UAV 15 use for agricultural purposes. It also examined the application 16 of SLAM methods for precision agriculture practices in the 17 greenhouse. Khelifi and Butun have presented the basic con-18 cepts of SUAV localization in [29]. They provide a limited 19 overview of the localization techniques and systems used 20 by SUAVs and their functioning principles. Therefore, this 21 paper comes to complement the survey established in [29] 22 by presenting a wide selection of papers, especially those 23 from the last three years, so that all new developments in the 24 literature are presented, compared and discussed. In addition, 25 this paper provides preliminaries of localization and path 26 planning, which do not exist in [29]. Furthermore, the com-27 munication technologies used in SUAV are also presented. 28 Additionally, another set of outstanding issues related to new 29 trends and future research directions are highlighted.

29
Before analyzing all the different localization systems used in 30 SUAVs, it is important to first understand the basic concepts 31 and conventional techniques of localization as well as the 32 most popular wireless technologies used in the localization 33 process. Also, when talking about localization it is very 34 relevant to differentiate it from navigation and path planning 35 methods, which will also be explained briefly in the follow- 36 ing. 38 Localization specifies the system for calculating the position 39 of an object. Any position is systematically associated with a 40 reference system [30]. The process of obtaining the position 41 or location is called positioning and/or geo-referencing [31]. 42 However, a distinction is sometimes made between posi-43 tioning and localization. Positioning refers to the process of 44 determining the position of the object. Therefore, localization 45 relates to the estimation of the position of this object by the 46 reference system such as the infrastructure [11], [32]. In the context of SUAVs, the process of estimation of the position is 48 used to localize a target or another UAV [19], [3]. In general, 49 the estimation is based on the analysis of a measurable 50 quantity with a corresponding model of the system, that 51 describes the relation of this quantity compared to the desired 52 one [32]. Various approaches are proposed in the literature to 1 formulate the estimation problem and the observation model. imprecise models, and environmental conditions [32], [29]. 10 Thereby, to measure the distance between the target and 11 the receiver different methods have been proposed in the 12 literature [30], [32]. For instance, Received Signal Strength 13 (RSS) [30], Time Difference of Arrival (TDoA) [30], Time 14 of arrival (ToA) [33], Time of Flight (ToF) [34], Angle of 15 Arrival (AoA) [35], antenna array systems [36], [37], trilater-16 ation [35], triangulation [35], hyperbolic [32], hybrid [32], 17 etc. Several previous articles already contain detailed de-18 scriptions of these methods and their characteristics; please 19 refer to these articles for more details [30], [35], [32].

21
To understand how the localization systems work, it is very 22 valuable to firstly describe the mainly used technologies in 23 this process. Currently, there are numerous wireless localiza-24 tion technologies used by the UAVs/SUAVs, depending on 25 their environment type: 26 Inertial Navigation System (INS) [38] is an earlier navi-27 gation system that operates offline and has a triggered initial 28 precise position; the following positions are calculated from 29 that initial position by using a 3-axis accelerometer [39]. INS 30 was followed by Global Navigation Satellite System (GNSS) 31 later on [40], [41]. As mentioned in [41], GNSS is the most 32 accurate positioning system that is used on earth today. It is 33 not only utilized for the navigation and positioning of the 34 UAVs but also by almost any kind of moving vehicle for the 35 same purposes. There are various types as it is contributed 36 and used by various countries from all over the world, such 37 as the Global Positioning System (GPS) [41]. GPS is a 38 satellite-based navigation system that is employed in a wide 39 range of applications from mapping and vehicle navigation 40 to surveying. It is the most popular and worldwide known 41 system [41]. 42 Before GPS systems were out there, cellular systems were 43 being used for localization purposes [39]. Especially, cellular 44 systems of all generations 2G, 3G, 4G, 5G because they 45 provide higher data rates, more connectivity, better coverage 46 and a wider range of services [5], [41]. In addition, they are 47 very well suited with the triangulation method, as cellular 48 towers can serve as fixed positioned beacons for the target 49 location to be calculated [39]. 50 Usage of gravitational fields in the localization and naviga-51 tion purposes has been successfully used in the past. In [42], 52 the feasibility of using prior gravity anomaly measurements 53 that are 1 nautical mile apart for underwater simultaneous 54 localization and mapping is shown. According to authors' 55 finding, the prior gravity anomaly measurements, coupled 1 with the developed tools (such as using particle filters), pro-2 vided guidance to select optimal areas and missions the AUV 3 (autonomous underwater vehicles) could transit through to-4 wards minimal localization error at the goal location.

23
In another work [44], authors has shown another usage 24 of magnetic anomalies for indoor positioning algorithms, by 25 presenting a a publicly available dataset for the evaluations. 26 The dataset includes Inertial Measurement Unit (IMU) and 27 magnetometer measurements along with ground truth posi-28 tion measurements with an accuracy of cms.

29
Another interesting communication technology is Optical 30 wireless communications (OWC) [39]. It used the spectrum 31 of the light which requires direct Line-of-Sight (LoS) for 32 communication. Hence, limited by the physical (terrain, 33 landscape, etc.) and natural (fog, rain, etc.) obstacles. For 34 instance, Infra-Red (IR) [45], Laser scanner or LIght De- 35 tection And Ranging (LIDAR) [45] which is mostly used in 36 automated collision avoidance systems. It allows measuring 37 distances by illuminating the target with laser light and then 38 measuring the reflection with a sensor. Some other technolo-39 gies used Radio Frequency (RF) [39], [45] such as Ultra- 40 Wide Band (UWB), Radio Frequency Identification (RFID), 41 Bluetooth, Wireless Fidelity (Wi-Fi), ZigBee/Z-Wave, and 42 Low Power Wide Area Network (LPWAN). UWB is a radio 43 technology that exploits a very low energy level for short- 44 range, high-bandwidth communications over a large portion 45 of the radio spectrum. It has traditional applications in non- 46 cooperative radar imaging, as well as most recent applications, span target sensor data collection, and precision locat- 48 ing towards tracking applications. Owing to its very short 49 pulse duration, UWB is a promising technology for ultra 50 low power, precise ranging, and positioning applications. 51 Since GNSS signals are mostly blocked and useless indoors, 52 RFID has been offered to solve indoor localization [39] while 53 Bluetooth uses a 2.4 GHz radio frequency band and various 54 applications of it can be found in the smart home, health, and 55 sports industry.

56
As mentioned in [46], Bluetooth technology is employed 1 by a UAV system to operate in short-range wireless for 2 executing some tasks it is developed. The goal was to achieve 3 an autonomous flight that is capable of carrying on different 4 flight missions. Wi-Fi is an IEEE 802.11 based wireless 5 network technology [39] used in-flight control of UAVs and 6 real-time data transmission (such as photo, video, GPS data, 7 etc.) between UAVs and devices on the ground. ZigBee/Z-8 Wave is a low-energy wireless protocol based on IEEE 9 802.15.4 standard commonly used in home/office automa-10 tion, medical, and industrial applications that have low data 11 rate transmission, require long battery life, and need secure 12 networking [39]. 13 Meanwhile, to further increase the positioning accuracy 14 of UAVs, alternative solutions that exploit existing radio 15 transmissions such as WiFi, ultra-wideband (UWB), or cel-16 lular networks for trilateration or triangulation with antenna 17 arrays should be considered [37]. These advanced solutions 18 significantly improve the accuracy of the localization system 19 and reduce the required infrastructure [47].

20
In general, existing technologies yield advantages and 21 restrictions. Thus, it is difficult to create cost-effective yet 22 efficient localization solutions using a standalone technol-23 ogy [32]. Given the complementary characteristics of these 24 various technologies, the integration of multiple technologies 25 at once is becoming a trend to achieve reliable, continuous, 26 accurate, and fault-free localization.

28
Navigation is defined as the process of guiding a user or an 29 object to a target, usually by following a predefined path. 30 For this, it is necessary to continuously repeat the process of 31 localization and provide directions to follow a path or reach 32 the target. Therefore, path planning in the context of UAVs 33 is considered as following a trajectory (markers) in the air by 34 flying from one position to another (see Figure 2). Most methods of path planning use algorithms created for 36 the decomposition of an area of interest to assess the one that 37 VOLUME 4, 2020 should be used to give the optimal solution to determine the  Some pioneering methods such as RRT [48] and RRT * that can be used to auto-adapt the location and improve the 32 efficiency of maintaining this specific dynamic flight path. 33 Therefore, some works rely on estimating the motion of 34 UAVs to build the map by using only data from onboard 35 visual sensors such as cameras, LiDAR sensors, 3D sensors, 36 and so on [50]. While others opt for the Simultaneous Lo-37 calization and Mapping (SLAM) approach [51]. The latter, 38 allows a UAV to localize itself in an unknown environment by 39 merging different sources of information while moving and at 40 the same time building a map without any prior information. 41 Recently, machine learning (ML)/artificial intelligence 42 (AI) based approaches have been an attractive tool for UAVs 43 path planning [52]. They have a great ability to handle the 44 uncertainty present in the environment with low computa-45 tional complexity [53]. In addition, ML/AI methods provide 46 an adaptive structure that is easy to implement and which is suitable for SUAVs collision-free paths to safely reach 48 their mission goals as they efficiently model this complex 49 optimization problem. 50 In summary, as SUAVs become more autonomous, they 51 have to figure out feasible paths for the whole swarm to 52 operate at high levels of autonomy and without any human  Further, scalability is the key milestone towards the applica-9 tion of SUAVs in large-scale networks. As such, this paper 10 elaborated aforementioned important aspect in each category 11 that is presented.

13
Flying a swarm of UAVs simultaneously implies the strict 14 requirement to maintain a safe and controlled formation 15 distance between them during their missions. For this, they 16 must continuously adapt and estimate their positions accord-17 ing to the flight conditions and the surrounding environ-18 ment [4], [11]. Therefore, such constraints introduce signif-19 icant challenges that complicate their localization process 20 and that of the targets as well [19], [3]. Also, a distinction 21 must be made between absolute localization (i.e., relative to 22 a map) and relative localization (i.e., relative to other drones 23 in the swarm) during the position calculation process. In what 24 follows, several prospective techniques used in the literature 25 on UAV swarm localization are briefly outlined while the 26 state of the art related to the use of these techniques to com-27 pute their positions is highlighted. Additionally, five classes 28 are identified: Computer vision, measurement requirements, 29 Cooperative localization, Intelligent localization, and Place-30 based localization techniques.  Meanwhile, the fast growth and the proliferation of ex-37 ternal cameras, all types of sensors, and other embedded 38 UAV systems have led to greater volumes of real-time aerial 39 images and information under different environmental condi-40 tions (see Figure 3). Consequently, a significant enhancement 41 in the estimation of the UAVs' positions and mapping has 42 been demonstrated. 43 Further, according to the purposes of UAVs swarm tasks, 44 two main classes of computer vision can be distinguished 45 depending on their environment. The first is where the 46 environment is to be explored such as target recognition, 47 navigation, coverage, and mapping. The second is only to 48 be traversed or exploited. For example, crossing an obstacle 49 field with a prescribed goal or desired formation [2]. 50 However, the scalability of methods in these categories 51 is limited by the computational power, the quality of the 52 onboard sensors and cameras, as well as the energy con-53 sumption of the UAVs. Thus, such vision-based techniques 54 in all its rich complexity, is far from being an easy task.

19
Consequently, this subject has been a hot spot for research 20 and continues to attract attention.  Thanks to AOA (angle of arrival) measurements that depend 10 on the target-receiver distance, they adjust heading angles 11 for optimal swarm deployment. Moreover, this sophisticated 12 mechanism increases the network complexity and lead to a    must tolerate communication failures. The consensus theory, 10 according to which a team of UAVs controls the processing 11 system in a distributed way to converge towards a common 12 agreement and value, was introduced in [75]. In [75], de-13 centralized sliding mode controllers (SMCs) which allow 14 the members of SUAVs to reach a consensus in altitude, 15 handing, and angles were designed. The work proposed in 16 [76] tackled the problem of formation keeping by applying 17 the back-stepping and graph theory methods in cooperative 18 control between UAVs. Although this approach allows rapid 19 dynamic response and low tracking error when tracking the 20 virtual leader, it only takes into account the formation system 21 of the UAVs from their take-off to their assembly process, 22 without addressing the formation vortex effect, the desired 23 mission goal, the shape and the quality of each UAV.

24
As stressed in [77], when it comes to the large-scale 25 networks with limited centralization ability, it is not pos-26 sible to employ a centralized entity to perform joint real-27 time decision making for entire network. This introduces 28 the scalability challenges, while multi-agent reinforcement 29 shows the opportunity to cope this challenges and extend the 30 intelligent algorithm to cooperative large-scale network. tions. Yet, even though they provide huge benefits, a criti-4 cal constraint of these techniques is the arduous adaptation 5 process to dynamic conditions, as they have to handle the 6 growing amount of all training data which highly affects 7 their scalability. Added to that, these methods increase the 8 computational complexity and cannot be executed on board 9 due to the limited computational capacity to learn the model 10 and decide the appropriate policy actions.

11
The localization problem is formulated as a multidimen-12 sional optimization problem that can be solved using numer-   Localization techniques can also be grouped into two major 13 categories due to where the localization takes place; indoor 14 vs. outdoor. The most distinguished among these is the usage 15 of GNSS aid during the localization of the outdoors for pre-16 cise estimation. However, GNSS connectivity in congested 17 urban areas with a dense distribution of skyscrapers might be 18 challenging even for outdoor navigation. In the same manner, 19 localization is a real challenge for indoor environments. 20 Hence GNSS signals are blocked by the thick walls, concrete, 21 and steel, some other methods are used for indoor localiza-22 tion. For outdoor localization, other than very well-known 23 techniques/technologies, some other methodologies are also 24 being proposed. For instance, Hoshiba et al. authors designed 25 and implemented a UAV-embedded microphone array system 26 for sound source localization in outdoor environments. The 27 concept is based on exploiting sound information as an aid 28 for localization of the UAVs in search and rescue activities to 29 compensate for poor visual information [84]. 30 Indoor localization and navigation of UAVs constitute 31 a critical part for autonomous flight and automated visual 32 inspection of elements in continuously changing environ-33 ments such as construction sites. In [85], Kayhani et al. 34 discussed the implementation and performance assessment 35 of an Extended Kalman Filter (EKF) for improving the esti-36 mation process of a previously developed indoor localization 37 framework that have used visual markers.   Figure 6). In the following subsections, the major con-54 VOLUME 4, 2020 tributions proposed in the literature related to this subject are   in their localization process. Although these emerging tech-10 nologies bring new opportunities in many areas, significant 11 changes in the localization mechanisms of SUAVs are taking 12 place. In this section, a set of outstanding issues related to 13 new trends are summarized and discussed, as well as relevant 14 future research directions are highlighted.

15
• Resource Management in SUAVs Renewable energies 16 and green communication are promising solutions for 17 power-supplying and energy-saving SUAVs, which en-18 ables them to autonomously operate without any as-19 sistance and physical intervention. This presents great 20 potential for their localization system as they provide 21 a self-sustainable ecosystem by harvesting energy from 22 the environment and continuously producing and sup-23 plying power to their system [76]. Therefore, the auton-24 omy of the UAVs is no longer hampered by their energy 25 suppliers and it is affordable to improve the localization 26 through additional hardware such as sensors, cameras, 27 etc. Hence, future research must be directed toward 28 efficiently managing the restricted UAV resources by 29 designing hardware and technologies that enhance their 30 endurance, recharging batteries and improved their lo-31 calization precision [89]. On the other hand, it is also 32 essential to find the best trade-off in energy consumption 33 between SUAV members to increase their flight time 34 with the aim to extract as much information as possible 35 from the available observations to raise their localization 36 accuracy.

37
• Localization over 6G, B5G, 5G and telemetry. The 38 proliferation of mobile services via B5G/5G/6G tech-39 nologies will enable SUAVs to use localization via telco 40 providers rather than GNSS [90]. Usage of telemetry 41 data will be also seamlessly possible for the GCS, 42 by the dual telecommunication channel provided by 43 the telco services [91]. Moreover, mmWave and tera-44 hertz communication are new concepts that curtail the 45 spectrum problems and capacity limitations of current 46 communication. In the coming years, these emerging 47 technologies may make it possible to provide accurate 48 localization as they offer larger bandwidth and pervasive 49 high-speed access in complex environments. Therefore, 50 future SUAVs require to support all these leading-edge 51 technologies in the design of their localization systems 52 as they constitute the next generation of networks and 53 communication trends. 54 10 VOLUME 4, 2020 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and gration SUAVs for the delivery of medical supplies and 10 patients monitoring.

11
• Cyber-security and Blockchain. Cyber-security of in-12 formation systems has been a great concern in the last 13 decade. For instance, many IDS schemes have been 14 proposed not only for WSNs [99], but also for the recent 15 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2022.3181377 Ref.
[59] intelligent ML is based on YOLO object detection system Ref. [79] intelligent PSO optimal relative position Ref. [84] outdoor UAV-embedded microphone array system exploiting sound information as an aid for localization of the UAVs in search and rescue activities to compensate for poor visual information Ref. [85] indoor EKF improving the estimation process of a previously developed indoor localization framework that have used visual markers Ref. [86] indoor/ outdoor collaborative UAVs opportunistically share information whenever they are in communication range of each other to enhance overall consensus-ed data Ref. [87] infrastructure semi-autonomous GCS is equipped with a transmitter/receiver that transmits command messages and collects telemetry data to/from the managed UAVs. Ref. [88] FANET cellular communication a hybrid architecture of an infrastructure-based network making use of cellular wireless communications infrastructure and establishing network protocol between drones without intervention of a GCS. This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and content may change prior to final publication.  blockchain systems towards securing the internet of things," 2020.

20
List of abbreviations are listed in Table 3.