NSGA-II Based Joint Topology and Routing Optimization of Flying Backhaul Networks

This paper addresses the problem of optimizing the deployment of Flying Backhaul Networks (FBNs). The latter comprise Unmanned Aerial Vehicles (UAVs), which are used as access points to provide coverage to a set of ground nodes deployed in a target area. The optimization problem is addressed by means of a Multi-Objective Optimization Algorithm (MOEA), which calculates Pareto curves of UAV placement, providing different trade-offs between the considered objectives: (1) to minimize the number of UAVs, and (2) to maximize the Packet Delivery Ratio (PDR). The selected MOEA is NSGA-II. An embedded single objective Genetic Algorithm (inner-GA) is used to optimize routing, finding the paths that maximize the PDR. In order to obtain consistent solutions for the PDR taking into account MAC layer contention, the scheme makes use of an existing fixed-point algorithm (FPA). Simulation results were obtained for different scenarios combining average versus maximin PDR objective funtions, two different routing optimization algorithms, as well as single sink versus multiple sink traffic patterns.

The associate editor coordinating the review of this manuscript and approving it for publication was Xujie Li . lar network), such coordination may fail. For example, due 31 to the large areas to be covered or accessed, teams may not 32 be able to report their findings or subsequent strategic plan in 33 a timely manner, and satellite communication may not be an 34 option due the high cost of operation and/or lack of terminal 35 equipment. This is where FBNs may be decisive. 36 FBNs present the following advantages: firstly, UAV 37 deployment is faster than recovering a crippled cellular net-38 work. Secondly, the acquisition and operational costs are 39 lower. However, the design of such network is challenging 40 and should be done carefully. Finding a suitable UAV topol-41 ogy that meets the overall network Quality of Service (QoS) 42 requirements is a challenging task in an FBN [7]. Essentially, optimized 45 UAV placement can be achieved either using a centralized 46 or distributed scheme. In a typical centralized UAV scheme, 47 there will be a single entity responsible for processing GN 48 data and updating the UAVs with the new positions. This is 49 different from a distributed placement scheme, where UAVs 50 proposed an UAV placement scheme inspired on bird flock-106 ing. The aim is to maintain connectivity among UAVs, 107 while adapting to the mobility of the nodes on the ground. 108 In order to achieve this goal, each UAV should follow a set 109 of rules/behavior, namely, ''move to the point above ground 110 Centroid'', ''repel from UAV'', ''attract toward UAV'' and 111 ''random walk''. The rules are represented by a state digram, 112 and the transition from one state to another depends on 113 the distance between UAVs. The authors in [14] propose 114 a distributed mobility algorithm based on a Virtual Spring 115 Force (VSF) model, through which the UAVs self-organize 116 into a mesh structure by guaranteeing QoS over the aerial 117 link, and providing coverage to isolated GNs. This algorithm 118 was further developed in [8], where Connection Recovery 119 and Maintenance (CRM), as well as Mobility Prediction 120 (MP) mechanisms were also integrated. In [4], the authors 121 propose a Graph Convolutional Multi-Agent Reinforcement 122 Learning (MARL) method to maximize coverage of ground 123 nodes by UAVs. The convolutional input layers allow local 124 collaboration among neighbour UAVs, which is translated 125 into better coverage performance compared with Deep Q-126 Learning (DQL) running independently in each UAV. The 127 number of UAVs is fixed, and the reward function only con-128 siders the coverage score and energy spent by the UAV, not 129 taking into account MAC layer performance. 130 Centralized schemes rely on a single entity having full 131 knowledge of node positions, and control over the UAVs. 132 In this paper, we propose a centralized scheme to jointly 133 optimize UAV placement and routing in FBNs. Therefore, 134 the remainder of this section will focus approaches of this 135 kind proposed so far. In the past few years, this topic was the 136 target of a significant number of research works. The works 137 on centralized schemes may be divided in two groups. 138 The first group does not consider inter-UAV connectivity 139 (e.g., the UAV relay nodes are directly connected to terrestrial 140 base stations), or simply abstracts inter-UAV communication, 141 taking it for granted, so that the placement algorithm does 142 not have to bother with it. Galkin et al. [15], proposed small 143 cells mounted on UAVs to offload ground users from the 144 macrocell infrastructure. The K-means clustering algorithm 145 is used to optimally place the UAVs. In [16], Kalantari et al. 146 proposed a 3D UAV placement scheme using the Particle 147 Swarm Optimization (PSO) algorithm. Similarly to our work, 148 UAVs are used as flying access points, and the main goal is 149 to find the minimum number of UAVs and their 3D coordi-150 nates to service all users with some target QoS requirement. 151 Mozaffari et al. [17], use circle packing theory to deploy 152 multiple UAVs, in order to maximize the coverage area. 153 The paper [12] was a previous work by our team. Here, an 154 NSGA-II based scheme is used to optimize two different 155 objectives: (1) to maximize the fulfillment of the data rates 156 required by the GNs, and (2) to minimize the number of 157 UAVs. A scheme to reduce algorithm search space based 158 on the computation of the convex hull ( [18]) formed by 159 the GNs was proposed, which is also adopted in the present 160 work. The link budget is calculated based on a log-distance 161 path loss model. IEEE 802.11g data rates are considered, 162 and the data rate of a link is simply the highest among 163 those whose receiver sensitivity is lower than the received  and it must be considered in the optimization process.

216
The work presented in this paper belongs to this group. 217 Reina et al. [19] proposed an optimized deployment scheme 218 that uses Multi-Layout Multi-Population Genetic Algorithm 219 (MLMPGA) as the optimization technique. The network has 220 three main requirements that should be satisfied and bal-221 anced: it should provide coverage and redundancy, and it 222 should be fault tolerant. In order to achieve this goal, the 223 authors define a weighted multi-objective fitness function, 224 allowing the use of the single-objective MLMPGA. In [20], 225 the authors mathematically formulate the placement opti-226 mization of UAVs as a multi-objective problem and solve it 227 as bi-objective linear optimization model. In paper [21], The 228 authors propose a system named Traffic-Aware Multi-Tier 229 Flying Network (TMFN). A TMFN consists of a mobile and 230 physically reconfigurable network of Flying Mesh Access 231 Points (FMAPs) and Gateway UAVs organized in a two-tier 232 architecture, which is able to quickly readjust its topology 233 according to the traffic demands of the users. In order to con-234 trol the TMFN's topology, the authors propose a traffic aware 235 Network Planning (NetPlan) algorithm, based on the concept 236 of Potential Fields (PFs). Although the paper assumes a mul-237 tihop network architecture and tests the proposed algorithm 238 in such an environment with ns-3, the NetPlan algorithm only 239 takes into account the access links between GNs and FMAPs. 240 In [22], the authors present a scheme to explore a region 241 of interest where a terrestrial network is deployed, detect-242 ing holes in the network topology, after which an algorithm 243 optimizes the selection of spots for placement of patch UAVs 244 that will increase the communication performance. In [23], 245 the authors propose a topology construction and adjustment 246 scheme, where the optimal topology is built using a PSO 247 algorithm, while the adjustment is based on gradient descent 248 using the same performance metric. The two algorithms are 249 integrated so that PSO only runs when the edit distance 250 between the current graph and the previous one calculated 251 by PSO is high enough. Thus, there is a compromise between 252 the optimality of PSO and the computational performance of 253 gradient descent. There is no attempt to minimize the number 254 of UAVs, which is fixed, so the PSO is single objective. 255 Departing from the previous work, in [3], the same team 256 proposes a joint mission assignment and topology manage-257 ment scheme. The scheme comprises three algorithms. The 258 first one performs a global optimization of mission assign-259 ment (greedy), then PSO based router placement and routing. 260 The second algorithm locally adjusts relay UAV positions 261 before the difference (edit distance) relative to the initial 262 network is too large. The third algorithm performs mission 263 reassignment of UAVs. Both the second and third algorithms 264 try to avoid running the first one (i.e., global optimization), 265 in order to reduce computation complexity. In both works, 266 the network performance metric is based on link distance and 267 thus does not take into account MAC layer aspects. In [24], 268 the authors focus on cellular networks, proposing an heuristic 269 iterative algorithm to obtain a connected network comprising 270 ground terminal nodes and a ground control station, which 271 is achieved by means of a mesh network of UAV relay 272 nodes. The worst case complexity of the proposed algorithm 273 proved to achieve the minimum number of UAV relay nodes 275 or the respective optimal positions. Table 1 lists the related works described above, highlight-  There are several related works in Table 1 that consider 285 multiple optimization objectives (e.g., [19]). These works use 286 to reduce the multiple objectives to single objective optimiza-287 tion, by means of prioritization, weighted sum, product, etc.  The work [23] is the one that bears more resemblance 297 to ours in terms of system model and performance objec-  (e.g., [25], [26], and [27]). Detailed comparison between 319 alternative algorithms deserves a dedicated study, being out-320 of-scope of this paper.

335
Definition 1: A wireless network is said to be connected 336 when there is a path between every pair of nodes. Hence, in a 337 connected network all nodes are reachable.

338
Definition 2: Two or more nodes are said to be neighbors 339 when the Euclidean distance between each pair is shorter than 340 or equal to D.

342
The scheme proposed in this paper aims to find the best 343 trade-offs between the FBN cost in terms of the number of 344 UAVs, and the achieved PDR. In this section, mathematical 345 formulation of each objective function is presented. Similarly to [12], the present work assumes that there is a 348 cost associated with each used UAV. Thus, minimizing the 349 number of UAVs is desirable. This is done by restricting the 350 UAV coverage to the sub-area a ⊂ A that corresponds to 351 the convex hull (convex envelope) [18] formed by the GNs 352 in A. This restriction of the deployment area also reduces the 353 complexity of the algorithm. In order to further reduce that 354 complexity, a is discretized in a grid layout according to the 355 following relation: where is the 356 distance between two neighboring UAVs, which is adjusted 357 by changing µ. Let Q ⊂ a be the discrete set of allowed 358 UAV deployment points.

359
Let q j ∈ Q be the j th potential UAV placement point. Let 360 {δ u q j }, ∀u ∈ U , ∀q j ∈ Q be defined as a set of binary variables 361 indicating which points are currently being used by an UAV, 362 as follows: Let {ζ u v }, ∀u ∈ U , ∀v ∈ V be defined as a set of binary 365 variables indicating which GNs are being serviced by each 366 deployed UAV. It is assumed that a GN will be connected to 367 VOLUME 10, 2022 The first objective is to assign values to δ u q j , so that valid 370 solutions are found to the following problem:  if utilization is equal to one, some arriving packets cannot be 432 served, but the service rate for each path is still proportional 433 to its compensated arrival rate as given in the second line 434 of Equation (3). 435 From Equation (3), one can derive the fraction of incoming 436 traffic rate that is sent over each path. Let h − i,p represent the 437 node that precedes node i in path p. The arrival rate from 438 path p at node i is calculated as follows: (4) 440 This is obviously not valid when node i is the first node 441 (i.e., the originator node) of the path, in which case λ i,p is 442 set as an input parameter of the algorithm. A comprehensive 443 explanation on how to compute β i,p is provided in [10]. The 444 computation of T i,p is equal to the sum of four components 445 as follows:  In order to find a consistent solution for the parameters 455 k i,p , λ i,p , β i,p and T i,p , the scheme uses the FPA equations as 456 provided in [10]. The FPA structure and stopping conditions 457 are adopted from [29]. 458 In order to compute the network PDR at the destination 459 node(s), represented by T , the scheme considers a set of active 460 connections in the network, denoted by C, and the set of 461 paths used in connection c ∈ C, denoted by P c . Two metrics 462 are considered, corresponding to alternative PDR objective 463 functions:

464
(i) Average PDR. This metric is computed as follows: (ii) Minimum PDR. The PDR provided in path p i , denoted 472 t p i , in connection c ∈ C is calculated as follows:  and crowding-distance-assignment (F i ) correspond respec-509 tively to the the Pareto ranking and diversity preservation 510 procedures described above. R t has size equal to 2N , being 511 formed by combining parent S t and offspring Z t populations.

512
F i refers to the i th front or level.

517
In order to find optimal solutions, NSGA-II must be able to 518 compute the values of two objectives from the above repre-  to find the best routes from source GNs to the destination 525 GNs, which lead to a higher value of PDR for the current 526 placement of the UAVs. The value of the second objective 527 will be the highest PDR found by the inner-GA, which cor-528 responds to the best set of routing paths for a given NSGA-II 529 individual. Fig. 2 depicts the idea behind the computation of 530 the objective functions. The inner-GA will have its own set of parameters as pre-534 sented next: Given the UAVs' positions from the NSGA-II 535 chromosome, the inner-chromosome will be a set of hash 536 tables of variable size, each of which representing a flow from 537 a source GN to a destination GN. For a given set of UAVs 538 occupying specific positions in the network, a source GN may 539 be able to reach the same destination GN through different 540 paths. Therefore, the scheme uses those paths to distinguish 541 different inner-chromosomes as they would represent differ-542 ent flows. As already stated, the communication between 543 GNs is realized through UAVs in a multihop fashion. Fig. 3 544 and Fig. 4 show a hypothetical communication network with 545 the flows of data and the corresponding inner-chromosome 546 representation, respectively.

547
Each key in the inner-chromosome represents a unique ID 548 of a node in the network, and the stored value corresponds to 549   may be different from each other depending on the number of 577 deployed UAVs to cover all GNs. In this study, the size of each 578 chromosome and the best solution found by the inner-GA 579 respectively correspond to the number of UAVs and PDR in a 580 given topology. The NSGA-II selection operation uses binary 581 tournament selection based on the crowded-comparison oper-582 ator ≺ n , in order to choose the best chromosome following 583 Equation (10). Here, given two solutions with differing non-584 domination ranks, NSGA-II prefers the solution with the 585 lowest (i.e., best) rank. Otherwise, if both solutions belong 586 to the same front, then NSGA-II prefers the solution that is 587 located in a less crowded region.

588
The genetic operators (crossover and mutation) are imple-589 mented in a way similar to our previous work [12], where 590 the crossover (with the probability p c ) between two chromo-591 somes is performed by finding a midpoint in a and drawing 592 (diagonally in 45/-45 degrees or horizontally or vertically) a 593 cutting line to divide the area in two parts in each chromo-594 some. Next, the operator removes all UAVs that are within 595 1 2 D distance radius along the cutting line within a . If the 596 separation line is either diagonally or vertically drawn, the 597 leftmost part of one parent is joined with the rightmost part 598 VOLUME 10, 2022 when the chromosome has variable length -as is the case of 627 the proposed scheme -, though a worst case characterization 628 is possible.

629
The proposed topology optimization scheme has a nested 630 algorithm structure, in which NSGA-II forms the outer layer. 631 One of the NSGA-II objective functions (PDR maximization) 632 is particularly complex, making use of the inner-GA to find 633 the best routing, i.e., the one that maximizes the PDR for a 634 given UAV topology. In turn, the PDR objective function of 635 the inner-GA is calculated by means of the FPA proposed 636 in [10]. It is in this nested objective function that resides 637 most of the time complexity of the algorithm. The num-638 ber of UAVs (|U |) corresponds to the size of the NSGA-II 639 chromosome, and significantly affects the performance of 640 the FPA. As already seen, since |U| constitutes one of the 641 NSGA-II objective functions, it is a variable, making it more 642 difficult to estimate the complexity of NSGA-II. As such, 643 in the following analysis it is considered that the number 644 of UAVs is fixed and equal to |U | max , corresponding to the 645 maximum allowed number of UAVs (an input parameter of 646 the algorithm).

726
In order to assess the impact of having more than one 727 sink/destination node, these scenarios were simulated with 728 two different traffic patterns:

736
In the UAV placement algorithm presented in [12], the use 737 of different altitudes (in the altitude range of [40,120] meters) 738 did not affect the performance, as the communication range 739 was quite long when compared to the maximum allowed 740 flight altitude. Based on these results, and since this work 741 does not address altitude optimization, a fixed flight altitude 742 of 80 m is assigned to the UAVs. Table 2 shows the summary 743 of the global/default parameters.

744
As already stated, an important and challenging step when 745 designing a GA is defining the stopping criteria, i.e., the point 746 where the algorithm should stop executing. There have been 747 several studies on the subject [30], [31]. Differently from the 748 VOLUME 10, 2022 present proposal, most of these studies consider fixed size and  As such, the stopping criteria is set as a fixed number of 758 generations. In order to determine the number of generations, 759 multiple simulations were performed to adjust the NSGA-II 760 parameters, such as p c , p c inner , p m , and p m inner to avoid exces-761 sive computation, as well as premature convergence. Table 3 762 shows the summary of the NSGA-II and inner-GA parame-763 ters. Parameters such as population sizes and genetic operator 764 probabilities were not optimized and thus can be a subject of 765 further studies.

766
In each scenario, and for each load value taken from 767 the set L, 25 runs of NSGA-II with different seeds, 768  Fig. 13). This only applies to load values above 830 30 kbps, since the latter leads to a PDR of 100% in all cases.

831
The PDR tends to decrease as the load increases, which 832 is also observed in Fig. 12 and Fig. 13. This complies with 833 the expected behavior, since higher load causes additional 834 contention at the MAC layer.  q j 1 = (500, 500), corresponding to opposite corners in the 842 deployment area.

843
In Fig. 14 (Scenario (A)), and differently from the sin-844 gle destination scenario (see Fig. 10), the results show an 845 increase of the minimum PDR (not an objective function in 846 Scenario (A)) as the average PDR increases, i.e., the previ-847 ously observed conflicting behavior is not present. This is 848 justified by the fact that data traffic is now split between two 849 destination nodes located at opposite coordinate points in the 850 target area. This configuration has the following advantages: 851 (i) Reduces the chance of having some nodes relaying data 852 belonging to many different flows, thus avoiding poten-853 tial bottlenecks;

854
(ii) Potentially reduces the degree of source-destination star-855 vation, as the number of immediate interfering neighbor-856 ing flows is reduced. This is a well known problem in 857 multi-hop ad hoc wireless networks, also known as Flow 858 In the Middle (FIM) problem [32]. In fact, the inner-GA 859 tries to find paths that suffer less from FIM, in order to 860 maximize the PDR.

861
In general, this leads to increased load balancing and thus 862 overall capacity, which is translated into higher PDR in both 863 metrics. The performance improvement gained by having one 864 additional destination node for each load value is shown in 865   Table 4. 866 Similarly, in Scenario (B) (see Fig. 15), the results show  The previous results allow the conclusion that having mul-874 tiple destinations placed apart from each other is beneficial 875 regarding the PDR.    simulations in different scenarios (see Fig. 18 and Table 6 for 890 clarity) was measured. In general, simulations with multiple 891    destinations take less time to converge when compared to the 892 single destination scenario. This is due to the reduction of 893 inter-flow interference, leading to faster FPA convergence.

894
Recall that FPA intialization assumes no contention between 895 flows, which is a closer guess when load is low. It is also 896 FIGURE 18. Average simulation time for the given simulation environment.
observed that maximizing the average PDR converges rela-897 tively faster than maximizing the minimum PDR.

899
This paper presents a joint UAV placement and flow routing 900 optimization scheme based on a nested architecture, where 901 NSGA-II algorithm forms the outer layer, while a single 902 objective GA is used as inner layer. Link performance is esti-903 mated by an FPA taking into account MAC layer contention. 904 Significant insights were only possible due to this feature. 905 Two scenarios, each with two optimization objectives, were 906 considered: (1) minimization of the number of UAVs while 907 maximizing the average PDR, and (2) minimization of the 908 number of UAVs while maximizing the minimum PDR. From 909 the simulation results, it was observed that the proposed 910 algorithm is able to provide meaningful Pareto curves, deter-911 mining a set of non-dominated solutions for UAV placement. 912 Based on the latter, the decision-making entities can choose 913 the one that best fits the application conditions and mission 914 management strategy at hand. Additionally, the performance 915 of the inner-GA routing optimization was compared against 916 the Dijkstra shortest path routing. The results show that the 917 inner-GA achieves better performance due to its capacity 918 of performing load balancing to reduce MAC contention. 919 This advantage is even more significant in complex scenarios 920 comprising more than one sink node. The simulation perfor-921 mance was also studied. It was concluded that scenarios with 922 multiple sink nodes tend to reduce flow interference, speed-923 ing up convergence of the FPA algorithm. In general, the 924 average PDR objective function leads to faster convergence 925 than maximin PDR.

926
In future work, we intend to evaluate the proposed scheme 927 when replacing NSGA-II and inner-GA with other alternative 928 algorithms. We are currently investigating alternative prob-929 lem formulations that optimize UAV displacement vectors 930 instead of positions, making use on GN movement prediction. 931 Finally, we plan to reduce the overall complexity of the 932 scheme by performing heuristic based adjustments to UAV 933 VOLUME 10, 2022