Collision Avoidance Method for Self-Organizing Unmanned Aerial Vehicle Flights

Autonomous unmanned aerial vehicle (UAV) swarm flights have been investigated widely. In the presence of a high airspace density and increasingly complex flight conditions, collision avoidance between UAV swarms is very important; however, this problem has not been fully addressed, particularly among self-organizing flight clusters. In this paper, we developed a method for avoiding collisions between different types of self-organized UAV clusters in various flight situations. The Reynolds rules were applied to self-organized flights of UAVs and a parameter optimization framework was used to optimize their organization, before developing a collision avoidance solution for UAV swarms. The proposed method can self-organize the flight of each UAV swarm during the overall process and the UAV swarm can continue to fly according to the self-organizing rules in the collision avoidance process. The UAVs in the airspace all make decisions according to their individual type. The UAVs in different UAV swarms can merge in the same space while avoiding collisions, where the UAV’s self-organized flight process and collision avoidance process are very closely linked, and the trajectory is smooth to satisfy the actual operational needs. The numerical and experimental tests were conducted to demonstrate the effectiveness of the proposed algorithm. The results confirmed the effectiveness of this approach where self-organized flight cluster collision avoidance was successfully achieved by the UAV swarms.


I. INTRODUCTION
Owing to scientific and technological development, the roles of unmanned aerial vehicles (UAVs) have become increasingly important in both military and civilian fields. The inherently small and flexible nature of UAVs allows them to perform well in many areas, and they are particularly useful in environments and situations that are not suitable for humans. UAV swarm collaboration will become commonplace to meet the increasingly complex requirements of future missions. Communication and control become increasing difficult as the number of UAVs increases; thus, it is important to achieve self-organization behavior during UAV flights. In self-organizing cluster flights, communication and messaging can occur between the UAVs; however, external instructions cannot be sent. Thus, individual UAVs must analyze the available information and make their own decisions. However, a swarm of UAVs The associate editor coordinating the review of this manuscript and approving it for publication was Haluk Eren. might merge from different directions during a self-organized UAV flight. Therefore, a collision avoidance method is required to prevent collisions assuming self-organized flight conditions. Improvements and optimization are required for the UAV self-organizing flight model, and a collision avoidance method is needed for UAV swarms comprising multiple UAVs in the self-organized flight mode. Many methods are currently employed for self-organizing UAV flights, such as the Reynolds rules [1], pigeon flock algorithm, and goose swarm algorithm. In the field of UAV collision avoidance, the approaches employed include the artificial potential field method [2], [3], ant colony algorithm [4], [5], genetic algorithm [6], [7], particle swarm optimization algorithm [8], [9], Markov decision method [10], and dynamic programming method [11], [12].
UAV collision avoidance methods have been studied for decades and various strategies have been proposed for collision avoidance by self-organized drones, but each method has practical difficulties. Thus, to facilitate collision avoidance in self-organizing UAV flights in practical applications, we developed a method by optimizing the parameters of the force formula used in the Reynolds rules. In the following, we review relevant research into the UAV self-organized flight collision avoidance problem. Table 1 lists 14 previous studies that proposed models for UAV self-organized flight collision avoidance, where the key characteristics of these studies are indicated as: physical experiment, target parameter optimization, UAV type, and leader agent based. In particular, ''physical experiment'' indicates whether a physical experiment was conducted, ''target parameter optimization'' indicates whether parameter optimization was performed based on the flight target, ''UAV type'' denotes the type of UAV, i.e., fixed-wing UAV(F) or quadrotor UAV(Q), while ''/'' indicates that the type of UAV was not specified, and ''leader agent based'' shows whether there was a leader in the UAV swarm.
Qiu and Duan [14] proposed a distributed optimization control framework for UAV clusters, which transformed the multi-objective optimization problem into a problem that can be solved using a single UAV. A distributed cluster control algorithm for UAVs based on an improved multipath I/O was proposed, which allows a UAV swarm to exhibit stable flight in a complex environment. Di et al. [15] proposed a two-layer control framework to address the collaborative surveillance problems using multiple UAVs, and distributed back horizon optimization for planning UAV movements in cluster flight situations. Braga et al. [16] used the Reynolds cluster rules to drive a swarm of UAVs, and then conducted physical experiments to demonstrate the feasibility of the method.
Kownacki et al. applied the two basic Reynolds rules and combined them with leadership features to achieve the collective flight of fixed-wing UAVs. Clark and Jacquesva [18] conducted a UAV cluster flight test at NASA's Dryden Flight Research Center as an important step in the development of a deployable distributed boid system. Sajwan et al. [19] clustered the UAVs in a leader-follower manner and solved each follower's control problem in the context of stochastic optimal control, where the problem was solved offline by dynamic programming to minimize the expectations within a limited range. Qiu and Duan [14] designed a distributed cluster control algorithm based on the dove cluster and a coordinated obstacle avoidance model for managing a situation where a UAV swarm flies through an environment containing obstacles without additional information.
Hauert et al. [22] studied the trade-off between the communication range and flight dynamics in a simulation within the Reynolds cluster. Hildenbrandt et al. [23] combined conventional coordination rules based on separation, attraction, and alignment with the details of ostrich behavior to provide new insights into the complex clustering mechanisms employed by ostriches and other birds. Han et al. [24] proposed a new concept called ''soft control'' for regulating the collective behavior of self-organizing, multi-agent systems and demonstrated a natural method for intervening in distributed systems. Vásárhelyi et al. [25] solved the problem of seamless navigation by a UAV swarm in a narrow space by addressing and resolving issues in terms of constrained motion and communication capabilities, delays, disturbances, or obstacles. They proposed carefully selected sequence parameters and an evolutionary optimization framework for a fitness function, and they used 30 self-organizing UAVs in field experiments. This was the largest airborne outdoor system without central control reported in previous studies, and they demonstrated successful avoidance by clusters of collective UAVs with effective avoidance control.
In the present study, we developed an optimized, selforganizing flight method to ensure stable flight and collision avoidance by UAV clusters. Our approach does not require a leader to self-organize the flight and it achieves self-organized flight collision avoidance for multiple UAV swarms by optimizing the resulting trajectory for fixed-wing UAVs and quadrotor UAVs, which is difficult for other methods. Our method does not require a leader, which means that there is no mutual control between the drones and a flight decision is made by assessing the airspace situation. The core idea of this method involves optimizing UAV flight parameters by defining a fitness function for optimizing UAV flight. We studied the collision avoidance method in a self-organized UAV swarm flight and designed a new collision avoidance algorithm for self-organized flight collision avoidance. The method was validated in simulations using MATLAB. VOLUME 7, 2019 The remainder of this paper is organized as follows.
In Section II, we present the traditional Reynolds flocking model. In Section III, we describe the optimization method based on the Reynolds rules and a collision avoidance method for self-organizing flight. In Section IV, we present the simulation results and their analysis. We give our conclusions and discuss future research in Section V.

II. TRADITIONAL REYNOLDS FLOCKING MODEL
It should be noted that we do not consider the specific flight control problem for the UAV and our method only plans the trajectory of the UAV. Thus, we address the collision avoidance problem for UAV clusters with normal communication between UAVs regardless of the time delay for communication between UAVs.
We write U =< P, V , A > to represent the UAV cluster system. P = ( p 1 , p 2 , · · · , p i , · · · , p N ), p i ∈ R n are defined as the position vector of UAV i ; V = ( v 1 , v 2 , · · · , v i , · · · , v N ) are defined as the speed vector of UAV i ; A = ( a 1 , a 2 , · · · , a i , · · · , a N ), a i ∈ R n indicate the acceleration vector of UAV i . The formula for the motion of a UAV is as follows.
According to this formula, the control of the motion by a UAV cluster may depend on the acceleration control. The speed and acceleration of the UAV are constrained by the following formulae: , where v max is the limited maximum speed value and a max is the limited maximum acceleration value. During the actual flight of an UAV, the change in flight angle needs to be limited owing to the flight performance limitations of the UAV.
The amount of change in the angle of an UAV is also limited, as shown in Eq. (4), where θ max is the maximum change in the angle of the UAV.
Reynolds first introduced a distributed flock behavior model in 1987 [1]. This model is based on the phenomenon of cluster flights according to the behavior of bird flocks. Subsequent research into flock modeling led to the definition of three simple rules for simulating flight decisions and state updates for a single individual in a flock of birds.
The clustering algorithm based on the bird flight rules proposed by Reynolds can be summarized as follows.
R1 -Alignment: individuals attempt to align their velocity to the average velocity of nearby flock mates.
R2 -Cohesion: individuals attempt to match the average position of nearby flock mates.
R3 -Separation: individuals attempt to avoid collisions with nearby flock mates.
However, the Reynolds rules are simply a simulation of flock behavior and model details need to be optimized further for application to a real UAV system.

III. OPTIMIZED FLOCKING MODEL WITH COLLISION AVOIDANCE
The proposed flocking model is based on the Reynolds rules and the optimized model is based on a previously described model [25]. As shown in Figure 1, self-organized UAV flight cluster collision avoidance is based on the Reynolds rules, where the first advance involves providing an update by calculating the combined effect of different forces. None of the different UAV swarms in the simulation are controlled by ground control stations, but instead they rely solely on information sent by other UAVs for analysis and decision making. In Section A, we introduce the basic Reynolds cluster rules and some added buffer functions to make the cluster rules more suitable for actual flight. In Section B, we introduce the parameter optimization method for the cluster flight rules as well as the design of the corresponding objective function and iterative search of a random simulation scene to find the various force parameters that best match the objective function. In Section C, we describe the self-organizing collision avoidance method for UAV swarms.

A. OPTIMIZED FLOCKING MODEL 1) REPULSION
To keep the focal UAV, UAV s , separate from the other UAVs, each UAV receives a collision avoidance force known as the separation force, which is defined as follows: where ψ i (t) is defined as: and n indicates the serial number of each remaining UAV other than the focal UAV (UAV s ). In Eq. (6), ψ i (t) is the nonlinear gain of pairwise repulsion and p s (t) − p i (t) is the distance between UAV s and UAV i . The symbol d sep defines the maximum interaction range at which UAVs start to repel each other and k is a coefficient that affects the magnitude of the repellent force.
2) VELOCITY ALIGNMENT UAV clusters are collectively managed by individuals matching their average cluster velocity according to Eq. (7): where v a,i (t) represents the speed of UAV i relative to UAV a and n represents the number of UAVs other than UAV a . The flocking algorithm needs to handle possibly large velocity differentials at the same time, so a braking curve known as a velocity decay function (denoted by D(.)) is applied to the flocking algorithm. The decay function aims to provide constant acceleration at high speed and exponential acceleration at low speed, as shown in Eq. (8): where r is the distance between a UAV and a desired stopping point, a denotes the expected acceleration, and p represents the maximum speed difference allowed at a certain separation distance.
To construct the velocity alignment pattern, the speed difference that can be tolerated at a certain relative distance is given by Eq. (9).
Thus, the velocity alignment term calculated for UAV i with respect to UAV j is: where v tor represents the amount of velocity slack required to allow for a certain amount of speed difference, which is independent of the relative distance, p tor 0 indicates the distance of UAV i relative to the stop point in front of UAV j , and k tor and a tor represent the linear gain and acceleration parameters for a pair of consistent speed changes, respectively. In Eq. (10), Q tor represents the linear coefficient of the speed fault reduction and v ij represents the absolute value of the speed difference between UAV i and UAV i .

3) FLOCK CENTERING
The UAV cluster needs to be as close to the center of nearby UAVs as possible, and thus the cohesion force is given by Eq. (11): where n represents the number of UAVs other than UAV c .

4) FLIGHT TARGET POSITION
To reach the desired position, the UAV cluster should know the general direction of the flight. We define the general VOLUME 7, 2019 direction as: where p des represents the target end point vector of the UAV.

B. FLOCKING OPTIMIZATION ALGORITHM 1) FLIGHT EFFECT EVALUATION FUNCTION
A statistic is employed to measure the anti-collision effect during the self-organizing flight of a UAV swarm, which is then organized according to the statistic for parameter optimization. Let T be the entire simulation duration, d ij (t) is the distance between UAV i and UAV j at time t, and d col refers to the danger zone around a UAV. In this situation, the collision risk is defined as shown in Eq. (16), where (·) is the Heaviside step function.
The next key element for self-organizing flight is the velocity alignment motion. The UAV cluster velocity correlation is measured as shown by Eq. (17).
Finally, we require that the UAV cluster move at a designated speed. Let v flock be the designated speed and it is measured according to Eq. (18).
The global function can be defined using the following three types of transfer functions.
The first type is a monotonically growing function represented as: where S(ξ, ξ 0 , d) is a sigmoid function: The second transfer function is a sharp peak transfer function, as follows.
Finally, we construct the following single objective fitness function, which includes all of the related flocking behaviors.

C. COLLISION AVOIDANCE BETWEEN SWARMS OF UAVs
When different UAV swarms merge together, the probability of collisions will increase significantly. The separation force defined above is no longer effective because its direction is almost opposite to the direction of speed. Therefore, new rules are necessary to achieve the anti-collision targets between different UAV swarms.
In the presence of other UAV swarms, the force responsible for preventing collisions is a force that overlaps with other forces as little as possible; otherwise, the force used for collision prevention would be diluted. To meet this requirement, we employ a three-dimensional (3D) geometry-based approach where we calculate the closest proximity position of the two opposing flying UAVs, and the corresponding applied force is then obtained according to the speed and position information for the UAVs. The magnitude of the force is the same as the previous separation force but its direction differs. As shown in Figure 2, the key step involves calculating the time taken to arrive at the closest point of approach (CPA). CPA occurs at the moment when the two UAVs arrive at their closest point. Algorithm 1 Self-Organizing UAV Flight Cluster Collision Avoidance Algorithm Input: initial position, target position, initial velocity of UAVs, number of UAVs N , simulation step length S, labels of the UAVs Output: trajectories of all the UAVs while flock do if label is the same &&D ij < D safe1 Steering_force = Repulsion_force + Alignment_force + Cohesion_force + Target_force -Current_velocity; elseif label is different &&D ij < D safe2 Steering_force = Repulsion_force; end Updated_velocity = Current_velocity + Steering_force; Updated_position = Current_position + Updated_velocity; end First, we calculate the CPA between UAV i and UAV j . The distance between UAV i and UAV j is defined according to Eqs (25)-(27): where p i = (x i , y i , z i ) and p j = (x j , y j , z j ) are the positions of UAV i and UAV j , respectively, and v We then determine the time required to reach CPA, which is defined as t. The formula p ij,t+ t · v ij = 0 holds when the UAVs arrive at the CPA. Assuming that the locations of the UAVs at time t + t are as given by Eqs (28) and (29): where t is the current moment, then we can calculate the relative position vectors for the UAVs at time t + t.
Therefore, the time interval t can be calculated as follows.
Based on the information described above, we propose a formula for calculating the anti-collision force for UAV i , as follows.
Similarly, the anti-collision force for UAV j can be written as follows.
The basic idea of this method is that the direction of the force is perpendicular to the relative position vector of the UAV when it reaches CPA. This approach gives the UAV the most effective control over the relative distance beyond the safe distance to the CPA. Algorithm 1 is the collision avoidance algorithm for a cluster of self-organizing UAVs, where the core step involves distinguishing the UAV repulsive force from different UAV swarms.

IV. SIMULATION AND RESULTS
Scripts were written in Matlab to generate four different simulation scenarios. Each UAV swarm was assumed to be heading toward its own target position along a straight line at 5 m/s. The minimum safe separation distance, d0, was defined as 15 m. The interval time, t0, for discrete points was set as 0.1 s. The corresponding weight coefficients w1, w2, w3, and w4 were calculated as 427, 0.225, 0.452, and 548, respectively. The maximum variation range for the pitch angle, A, was 0.74, and the minimum value, B, was −0.74. We assumed that each UAV could receive information from the other UAVs and that the simulation calculation times could be ignored. The simulations were performed using an HP EliteBook laptop, with an Intel i5 processor at 2.6 GHz and 4 GB of RAM.

A. UAVs IN A SWARM FLYING TOWARD EACH OTHER
We assumed that two swarms of UAVs approached each other from opposite directions, and thus collisions were imminent. In the simulations, each UAV swarm contained three UAVs and each UAV swarm was self-organized to form its own team VOLUME 7, 2019  at the target location. The initial state and target status for each UAV are shown in Table 2.
In Figure 3a, each UAV trajectory is displayed in the form of a 3D curve. The green tracks represent the trajectories of the first UAV swarm and the red tracks represent the trajectories of the second swarm. Figure 3a shows that the trajectories of the two swarms changed continually during the collision avoidance process. The UAVs in the two swarms flew from the scheduled starting position in a self-organized manner until the end of the flight. During the entire collision avoidance process, the two UAV swarms adaptively generated collision avoidance movements within the collision area, where the distributions of their positions in space were uniform and their trajectories satisfied the actual flight demands. Figure 3b shows the relative distance between each UAV pair. The minimum distance between any two UAVs was 15.03 m, which was above the lower limit of the safe separation distance.

B. UAV SWARM FLIGHT SCENARIO THAT MIGHT CAUSE A DOMINO EFFECT
A second scenario was generated to detect whether collision avoidance could be achieved in a high traffic density situation  where the airspace was limited and the UAV vertical separation distances were very small. Figure 4 shows that two swarms of approaching UAVs were present and their collisions were imminent. Unlike scenario 1, the UAV formations in each swarm were arranged in a vertical manner and this formation could lead to a collision domino effect when the UAV swarms met. Table 3 shows the initial state and the target status of each UAV.
The trajectory of each UAV is presented in the form of a 3D curve in Figure 4a. The green track represents the trajectory of the first UAV swarm and the red track represents that of the second swarm. In this simulation scenario, the UAVs generally adopted horizontal strategies for collision avoidance because vertical collision avoidance movements could have resulted in a domino effect. This scenario verified the anti-collision effect of self-organizing UAV clusters in a vertically stacked situation. Figure 4b shows the relative distances between each UAV pair, where the minimum distance between any two UAVs was 15.02 m, which was higher than the minimum safe separation distance.

C. UAV SWARM VERTICAL CROSS FLIGHT
This scenario was designed to test whether UAV swarm collisions could be avoided effectively in the vertical direction. Figure 5 shows that two UAV swarms approached each other at a vertical angle and collisions were imminent. The simulation showed that the method could readily ensure collision avoidance in this scenario and the anti-collision trajectories were very smooth, thereby meeting the realistic flight requirements. This scenario supported the effectiveness of the anticollision method when UAV swarms meet at a vertical angle. The initial state and target status for each UAV are shown in Table 4.
The trajectory of each UAV is represented by a 3D curve in Figure 5a. The green track represents the trajectory of the first UAV swarm and the red track is the trajectory of the second swarm. Figure 5b shows the relative distance between each pair of UAVs, where the minimum distance between any two UAVs was 15.04 m, which was above the minimum safe separation distance.

D. THREE DIFFERENT DIRECTIONS OF UAV SWARM GATHERING SCENE
This scenario tested the effect of the anti-collision method when several UAVs arrived in the same airspace from different directions. Figure 5 shows that three swarms of UAVs were flying from three different directions and collisions were imminent. This scenario tested whether the algorithm was still effective at avoiding collisions when the UAV swarms merged from three different directions. The simulation results showed that although the airspace density was high, the  anti-collision effect was very good and the flight paths did not require excessive deviations, thereby meeting the realistic flight requirements. The initial state and target status of each UAV is shown in Table 5.
The trajectory of each UAV is represented by a 3D curve in Figure 6a. The green track represents the trajectory of the first UAV swarm, the red track is that for the second swarm, and the blue track is that for the third swarm. Figure 6b shows the relative distance between each pair of UAVs, where the minimum distance between any two UAVs was 15.03 m, which was higher than the minimum safe separation distance.

E. SIMULATION ANALYSIS AND COMPARISON OF VARIOUS SCENARIOS
Each simulation scenario showed that all of the UAVs were located greater than the safe distance, thereby demonstrating that collisions were successfully avoided in the drone swarm. The relative distance maps indicated that the collision avoidance effect was very good for the UAVs, and each UAV swarm continued to move toward the target position after completing the collision avoidance process. The relative distances of the UAVs in the same swarm or in different swarms were also maintained in a smooth manner during the entire collision avoidance process. However, in the fourth scenario, the relative distances of the UAVs at the CPA varied more than those in the first three scenarios because the airspace traffic density was very high in the same airspace at the moment corresponding to the CPA, and each UAV swarm was at risk of colliding with the UAV swarm in both directions.
The mixed state of the UAV swarm was observed using the relative distance curves for the UAVs and by assessing the topology of the UAV swarm from the side. For each scene, two types of relative distance curves were obtained, where one tended to form a straight line and other tended to form a curve. The former represents the relative distances of the UAVs inside the swarm and the latter within the different swarms. The relative distance curve inside the swarm was smooth and it varies little, and thus the UAVs inside the swarm had a smooth and compact topology throughout the simulation.

V. CONCLUSION AND FUTURE WORK
In this study, the self-organized flight of UAV swarms was optimized based on the Reynolds rules. A new fitness function was proposed and the parameters were optimized using a CMA-ES optimization algorithm. A new cluster anticollision algorithm was also proposed. The simulation results proved that the proposed self-organized UAV swarm flight collision avoidance method performed in a safe and effective manner with smooth avoidance trajectories, thereby meeting the realistic flight requirements. The main contributions of our study are summarized as follows. 1) Self-organizing flight rules were modeled for a UAV swarm based on the traditional Reynolds rules and suitable constraints were added. A fitness function was constructed to optimize the rules in the model to ensure that the self-organizing UAV swarm flight collision avoidance behavior was aligned with the actual flight requirements.
2) The traditional self-organizing flight rules are not effective for addressing the collision avoidance problem between UAV clusters, so we derived new anticollision rules between UAVs to improve the collision avoidance efficiency and obtain smooth avoidance trajectories. In the simulations, the UAVs completed their anti-collision maneuvers without changing speed, which is suitable for meeting the actual flight requirements. 3) Our proposed method was verified in four simulation experiments, where UAV swarms approached faceto-face as well as extreme collision avoidance by UAV swarms in the vertical direction, vertical angle convergence to allow anti-collision by UAV swarms approaching in the horizontal direction, and collision avoidance by UAV swarms approaching the same airspace from three different directions. In future research, we will consider factors such as communication delays to further optimize the rules model as well as the drone speed to make the model more flexible. In addition, actual flight experiments will be conducted using fixed-wing UAVs to validate our proposed method.
YANG HUANG received the B.S. and M.S. degrees in information and communication engineering from the National University of Defense Technology, Changsha, China, in 2013 and 2017, respectively, where he is currently pursuing the Ph.D. degree with the Science and Technology on Information Systems Engineering Laboratory. During the B.S. degree, he studied the system engineering and the operation research. After finishing the B.S. degree, he began to study the selforganizing flight theory of UAVs and the collision avoidance problem of UAV clusters. His research interests include the air traffic management and unmanned aerial vehicle systems, such as collision avoidance of multi-UAV systems, especially the self-organized UAV collision avoidance problem.
JUN TANG is currently an Assistant Professor with the Science and Technology on Information Systems Engineering Laboratory, National University of Defense Technology. He was dedicated to the Ph.D. researches with the Technical Innovation Cluster on Aeronautical Management, Universitat Autònoma de Barcelona. His research interests include logistic systems, causal modeling, state space, air traffic management, and discrete event simulation. He is the winner of William Sweet Smith Prize, in 2015. He has been very active in the simulation community, organizing as the General Co-Chair for several international conferences. He acted as one of the main participants playing an important role in the FP7 European Project INnovative TEchnologies and Researches for a new Airport Concept toward Turnaround coordinatION (INTERACTION) with collaboration of Airbus.
SONGYANG LAO received the B.S. degree in information system engineering and the Ph.D. degree in system engineering from the National University of Defense Technology, Changsha, China, in 1990 and 1996, respectively. He joined as a Faculty Member with the National University of Defense Technology, in 1996, where he is currently a Professor with the School of Information System and Management. He was a Visiting Scholar with Dublin City University, Ireland, from 2004 to 2005. His current research interests include image processing and video analysis, and human-computer interaction. VOLUME 7, 2019