UAV Formation Control Under Communication Constraints Based on Distributed Model Predictive Control

In this paper, an anti-disturbance control for UAVs formations in a three-dimensional environment using relative information is proposed. To accurately describe the UAV cluster motion error in a three-dimensional environment, an error estimation model based on the UAV model is established to realize the UAV state estimation based on relative information. The control law to achieve the stability of the UAV formation under bounded perturbation is also derived using the backstepping control strategy. Next, to realize that the UAV formation can achieve optimal control under performance constraints, an algorithm based on model predictive control is proposed, and an ant colony algorithm is used to accelerate the solution process. Then, the stability of the UAV formation under the action of this algorithm is investigated. Finally, the superiority and effectiveness of the algorithm compared with the traditional backstepping control method are verified through comparative simulations.


I. INTRODUCTION
As an unmanned operational intelligence capable of multiple functions, Unmanned Aerial Vehicles (UAVs) are widely used in military and civilian applications due to their low cost, ease of integration and modular design [1], [2], [3], [4], [5]. UAVs formations are gradually replacing single UAV in complex missions due to their greater endurance and ability to handle complex missions compared to single UAV. The key technology for UAVs formations to achieve complex missions is formation cooperation control.
Currently researchers have studied more formation control for the case with complete communication links. Leader-Follower [6],virtual structure [7], behavior-based [8], Consensus-Based [9] and other formation strategies are widely used for formation control with communication links. [10] designed formation controllers based on second-order kinematic models using feedback linearization as well as The associate editor coordinating the review of this manuscript and approving it for publication was Zhenbao Liu . sliding mode compensators to design robust adaptive controllers to deal with parameter uncertainties during formation control while ensuring zero dynamic errors in the system. In [11], Zhao et al. used model predictive control to achieve formation control of rotorcraft UAVs by fully considering multiple constraints in the unmanned formation flight process. In [12], in order to expand the number of formations, a virtual structure-based control law was designed to realize operational experiments of large-scale formations of UAVs by a single operator. And Jonathan designed formation control strategy for inter-intelligent body damping in order to eliminate the effect caused by inter-aircraft damping in the literature [13] and implemented hardware-in-theloop formation control simulation. To further eliminate the interference of unmodeled dynamics for the system and improve the formation tracking performance, [14] designed neural adaptive sliding mode control controller to implement formation control.
However, changes in the UAV formation mission environment or mission conditions can cause damage or even unavailability of the communication link. Therefore, it is important to study vision-based formation control. In the past decade, there have been many vision-based formation control studies [15], [16], [17], [18], [19], [20], [21]. However, in [15], [16], [17], and [18], the formation height variation was neglected and only the formation control in two-dimensional planes was studied, and the formation control for complex nonlinear models in three-dimensional space was not considered enough. Although [19], [20] consider the formation control problem in a 3D environment, formation control in a perturbed environment has not been sufficiently studied in [19] and [20]. In addition, [21] and [22] also considered formation control in a 3D environment, but only the formation of three UAVs was simulated and the control of followers who could not observe the state of the pilot aircraft was not sufficiently considered. Although the above work has considered the UAV formation problem in vision-based situations, the formation control problem for multiple UAVs in complex environments has been less studied.
On the other hand, fixed-wing multi-UAVs need to achieve formation control in a short period of time when performing missions. In contrast, model predictive control(MPC) can achieve optimal control in the predicted time domain and has been widely used for formation control [23], [24], [25], [26]. [23] used single-layer recurrent neural networks and two-layer recurrent neural networks to achieve formation control in ideal and perturbed environments,respectively.In contrast, [24], [25] used neural dynamic optimization model predictive control based on [23] to achieve formation control for nonlinear models. [26] differs from the above methods in that the state variables are constrained to a constant design tube through dual time-scale neuro dynamic optimization, aided by state feedback gain scheduling. However, all the above strategies based on model predictive control to achieve formation control have an explosive computational burden with the increase of parameters as well as the prediction time domain.
Inspired by the above work, the formation control problem of multiple UAVs in a jammed environment and without communication is studied. A distributed model predictive control algorithm based on the backstepping control method strategy is designed to ensure that the followers can follow the virtual navigator to form a predetermined formation; and an ant colony algorithm is used to reduce the solution time. The main contributions are as follows: 1) Although the formation control problem of multiple UAVs was studied in [6], [12], and [14], the state information of UAVs can all be transmitted by a stable communication link, and the formation control designed in this paper is able to achieve formation control based on vision only.
2) Unlike [14], [23], [24], [25], [26] which use neural networks to estimate model uncertainty or solve the control signal, the paper uses the backstepping control method to design the control law and uses ant colony algorithm to solve it in the prediction time domain, which effectively reduces the computational burden and solving time.
3) The formation control algorithms designed in [21] and [22] all require the pilot to be within the follower's field of view, which implies that the state of the pilot is known. The formation controller designed in this paper only needs to track the state of the UAV in the neighborhood to achieve formation control.
4) Compared with [21] and the authors' previous work [27], this paper achieves formation control of multiple UAVs in complex environments with wind disturbances.
The rest of the paper is organized as follows. Section 2 establishes a stand-alone motion model of UAV, and the neighborhood UAV state estimation model. In section 3, we derive a three-dimensional error state model through the stand-alone motion model and design the DMPC controller and give the algorithm process. And then, we present the analysis process of the sufficient conditions for the convergence of the single-machine error model and the stability of the formation system. Section 4 conducts simulation and numerical analysis of three examples through MATLAB. Finally, in section 5, we draw conclusions and give suggestions for future work.

II. PROBLEM MODELING
In order to introduce the UAV formation movement and modeling process, we introduce three coordinate frames I, V i , L i , and as the inertial coordinate frame,the follower velocity frame of the UAV i, and the line of sight frame. The origin of the follower velocity frame and the line of sight frame is the mass point of the UAV. The relative relationship between the coordinate frames is shown in Figure 1.

A. SINGLE UAV MOTION MODEL
Assuming that there are N v fixed-wing UAVs in the formation, the dynamics between UAVs are decoupled, the influence of the roll of the UAVs is not considered, and the influence of the external wind effect is ignored. By considering v V i i = v i 0 0 T as the follower velocity vector with is the rotation matrix of I with respect to V i , v i is the speed of the i th follower UAV, and χ i is the yaw angles of follower UAV, and γ i is the pitch angle of follower UAV. p i = x i y i z i T is the position vector of the UAV in the inertial coordinate frame. By denoting D I v i as the velocity differential in inertial frame, ω V i i = ω xi ω yi ω zi T as the angular velocity vector in the follower velocity frame, as the acceleration vector in the follower velocity frame, we have the following Coriolis equation: By denoting x i = x i y i z i v i χ i γ i T ∈ R 6 as the states of followers, and by using (1) and (2), the motion equation of follower can be express as followers: By considering u i = a V i xi a V i yi a V i zi T ∈ R 3 as the input of the system, we have the expression of the UAV state space as follows:ẋ where f (·) is the corresponding nonlinear function.

B. NEIGHBORHOOD UAV STATE ESTIMOTION MODEL
To better describe the state estimation model, a schematic diagram of the relative relationship between the UAVs is first given, as shown in Figure 2.
By assuming l ij = l ij 0 0 T as the vector of position of UAV j with respect to line of sight frame L i , we have the position vector of UAV j in the intertial frame where By introducing t as sampling period, and the velocity of UAV j at time k can be obtained from equation (4) aŝ Then, the yaw angle of the UAV j can be expressed aŝ According to equation (6) and equation (7), the UAV pitch angle can be obtained aŝ According to equations (5), (6), (7) and (8), the UAV j state vector is obtained as The above model is the basic model for solving the UAV formation control, and the basic relationship between neighboring UAVs is constructed by the establishment of the UAV state estimation model. However, in practical application, it is also necessary to consider the error state relationship between UAVs and the design of the controller. Based on this, the error state space will be modeled and the controller design of the formation error system will be carried out in Section 2, in order to obtain the effective control of UAV formation in a limited time.

III. DISTRIBUTED MODEL PREDICTIVE CONTROLLER DESIGN
To address the problem that there is no accurate modeling of the three-dimensional spatial UAV formation error model in the existing literature, this chapter first establishes the UAV formation error model based on the UAV state estimation model in the previous chapter. After that, by introducing the error model, the design of the model prediction controller is carried out and the cost function based on the error model is given.
Without loss of generality, we give two assumptions of the algorithm as follows: Assumption 1: Only the leader can obtain the reference line. Follower UAVs can identify other UAV numbers, and obtain relative information of UAVs in the neighborhood without delay according to the information flow transmission topology.
Assumption 2: The algorithm only considers the effect of wind speed on the position of the UAV and does not consider the sensor error. And the wind speed is assumed to be bounded.

A. STATE ERROR MODEL
In order to better realize formation control, this paper firstly establishes the UAV error state model in three-dimensional.
By assuming x r = x r y r z r v r χ r γ r T ∈ R 6 as the reference state of UAV i , the error between UAV i and its reference state in the frame V i can be described as follows: Linearization leads tȯ e ir = Ae ir + Bu i + Hw (9) where, as shown at the bottom of the page. w = w 1 w 2 w 3 T is the bounded systematic error vector, i.e., |w| ≤ α 1 , |ẇ| ≤ α 2 , the exact form of which will be shown in the following.

B. FORMATION CONTROL DESIGN
The DMPC-based control design can be described as, at time k, the UAV i solves the optimal control problem with the predicted number of steps starting from the initial error e ir (k). Eq. (9) is discretized and described aṡ e ir =Ãe ir +Bu i +Hw (10) wherẽ where T is the sampling time and the value will be given in the simulation section.
In the UAV formation, we design different cost function for leader and followers,which can be expressed as follows: 1) cost function of leader as (11), shown at the bottom of the next page 2) cost function of followers as (12), shown at the bottom of the next page where x P = x T Px 1/2 represents the quadratic form of the vector. N f represents the set of followers. N i represents the set of UAVs in the neighborhood of the UAV i . e ir (k + l |k ) indicates the error state which the UAV i starts from time k and applies the control sequence u i (k + l |k ) in turn to obtain within the number of predicted steps. X i is the set of state constraints of UAV i which can be expressed as follows: U i is the input constraint set of the system,which can be described as follows: where a x max a y max a z max ∈ R 3 indicates the maximum value of control input in three directions and a ix a iy a iz ∈ R 3 indicates the maximum value of control input change in three directions. The weight matrix

C. ALGORITHM FLOW
We use leader-follower strategy to realize formation control and there is a fixed information flow topology in the formation. In order to distinguish the leader from other UAV in the formation, we use UAV 1 to represent the leader. The basic principle of the algorithm is shown in Figure 3.
The process of DMPC can be described as follows: Step 1: Bind UAV number and priority. Bind the reference to UAV1.
Step 2: At t = k, the UAV in formation are in a constant cruise state. Determine the UAV in the neighborhood, and measure the relative state quantities between the UAV. For the follower UAV: take the current state as the initial state of the UAV, and take the measured state quantity of the UAV in the neighborhood as the reference state quantity. Calculate the current time error as the initial state of rolling time domain planning, which can be described as follows e ir (k |k ) = e ir (t = k).
Step 3: Calculate the optimal input sequence U * i (k) which can be described as: (11) or (12). The first item of the optimal input sequence u * i (k |k ) is used as the input of the UAV at t = k+1.
Step 4: At t = k + 1, based on the new error state e ir (t = k + 1) of the formation, repeat Step 2.

D. SYSTEM STABILITY ANALYSIS
In the process of solving the UAV formation problem, the stability of the formation plays an important role, which not only reflects the performance of the controller, but also is an important guarantee for the UAV formation system to achieve the expected formation. The main idea of stability analysis is to find a suitable Lyapunov function and prove its degressivity. According to the above ideas, in the process of cluster stability proof, the following two problems need to be considered: one is the stability of the error state of the singlemachine system, and the coefficient relationship is given to ensure the stability of the single-machine system; the second is the stability of the formation system. In the following part, we first give and prove sufficient conditions for single-machine error stability. Then the cost function is used as the Lyapunov function of the system to prove the stability of the formation system.
Theorem 1: There exists k 1 , k 2 ∈ R, k 1 , k 2 > 0, such that when the virtual input satisfies the system (9) is asymptotically stable, where: Proof: From (9) we have We can rewrite the above equation in the following form: By assuming e 1d as the expected value of e 1 , we can describe the error δ 1 as follows: By taking the time derivative of the above equation. one can get:δ Choose the Lyapunov function as By considering (14) and (17), we havė SinceV 1 = δ T 1 ·δ 1 < 0, we have: where k 1 > 0, By assuming e 2d as the expected value of e 2 , we have The error δ 2 for e 2 can be expressed as: Substituting (19) and (20) into (18),we havė By substituting (14), (15) and (19) into (20), and taking the time derivative of it, we havė Let the Lyapunov function with δ 2 be By taking the time derivative of (23) and substituting (18) into it, we havė When taking δ T 1 · A 12 +δ less than 0 holds constant, and the system input can be found by substituting into equation (21) as (25), shown at the bottom of the page.
Considering that for the system (14), its ideal reference satisfies: e 1d =ė 1d = 0, e 2d =ė 2d = 0, substituting into (25),the simplification leads to the input as Theorem 1 gives a sufficient condition when the single machine error system is stable, and Theorem II is given below to prove that when the perturbation is bounded, using the cost function as the system Lyapunov function can make the system stable.
Proof: (1) Proof of equation (30) Solving the differential equation of (29), we have where: . . , 6} is the eigenvalue of the system (29) and L is the matrix of eigenvector sets corresponding to the eigenvalues. From Theorem 1, the system eigenvalue λ i < 0, i ∈ {1, 2, . . . , 6} holds, so when k → ∞, the following equation holds According to (15), (20) and e 1d =ė 1d = 0, e 2d =ė 2d = 0, equations (11) and (12) can be transformed into Firstly, equation (36) is solved and the sequence of optimal solutions of this equation is: The optimal state trajectory of the corresponding system is as well as the calculated error trajectory as ., N − 1 At moment k + 1, the solution of the construction problem (36) is Then, at moment k + 1, its corresponding state sequence as well as the error sequence are From the cost functions of the two moments, we have where ε i is the error term that can be adjusted according to the system accuracy needs, independent of time. So we have So at any moment, the system error state satisfies According to the above equation, when M 1 = i∈N v M i 1 at any moment the system error and the cost function satisfy (2) Proof of equation (37) From the proof (1), we have is a feasible solution of (36), then construct a feasible solution (38) in the predicted time domain N .
Then the cost function J * l_i 1 (k) corresponding to the optimal solution sequence satisfies Also for where, as shown at the bottom of the next page.
where ε i is a constant error term, independent of time. We have So the UAV error state system is consistently boundedly convergent.

IV. SIMULATION
In this section, the UAV formation is simulated. The simulation parameters are set as shown in Tables 1 and 2 below, and the initial states of the UAVs under the inertial system and the expected relative positions of each UAV to UAV1 are given in Table 2.
The desired formation is to form a diamond shape with distance units in m and velocity units in m/s. The number of simulations is 150 and the sampling time is 0.1 s. Assuming that according to the literature, for the problem of formation flight of UAVs in three dimensions, external disturbances can be assumed to have the same form as in that literature [28]. where an element value of 1 indicates a relative information interaction. To verify the superiority of the algorithms in this paper in accomplishing formation control and task retention, two algorithms are used for comparative simulation. Algorithm I, which relies only on inverse control to achieve formation and task retention (i.e., only equation (31) is used as an input to the system); and Algorithm II, which uses the complete algorithm of this paper.

A. PARAMETER COMPARISON
In order to compare the cost function and the convergence rate of the formation at different values of k 1 and k 2 , the values of the cost function and the error state plots are first given for k 1 and k 2 at different values.
From Fig.4 we can see that the cost function of the system has a great difference when k 1 and k 2 are taken at different values. In Fig. 5 and Fig. 6, the position errors as well as, k 1 and k 2 in Case4 are able to ensure that the position errors converge as soon as possible, but the errors in velocity, yaw angle and pitch angle converge more slowly. On the contrary, k 1 and k 2 in Case1 can ensure faster convergence in velocity, yaw angle and pitch angle, but slower convergence in position error. So using the appropriate k 1 and k 2 can make the system converge quickly with a small cost function.

B. STRAIGHT FLIGHT
In this section, the formation control effects of the two algorithms are simulated for comparison at uniform linear flight. The simulation results are shown in Fig. 7 to Fig. 12. Fig.7 and Fig.8 shows the position error, velocity error, and angle error of the trajectory-tracking UAV. It is obvious that there is a control time delay in the Backstepping Control, which leads to a slow convergence of the error and serious    jitter during tracking. Compared with the Backstepping Control, the algorithm proposed in this paper can effectively shorten the convergence time and reduce the tracking error, which can be attributed to the continuous optimization of the coefficients in the Backstepping Control by the MPC. As seen in the Fig.9, compared with the Backstepping Control, the algorithm in this paper can effectively achieve the convergence of the system in a shorter time and the input sequence produces less fluctuation. From Fig.10, we can see the algorithm proposed in the paper is able to achieve convergence of the system error while ensuring fast convergence   of the system. Fig.11 shows the flight trajectory of UAV in 3D space under the action of the algorithm in this paper, and the pentagram represents the virtual pilot. From the trajectory, the four UAVs can achieve a reduced inter-formation spacing, realize diamond-shaped formation flight, and have a better formation maintenance effect. In order to further compare the control effects of the two algorithms for formation UAVs, a variable radius formation flight is set. The solution time of each UAV shown in Fig.12 is less than the sampling time of 0.1s.

C. VARIABLE TURNING RADIUS FLIGHT
In this section, the formation control of the two algorithms is simulated for comparison during variable radius curve flight.
The simulation results are shown in Fig.13 to Fig.18.    It can be seen that the inverse control produces severe jitter for the tracking error of the trajectory and it is difficult to converge to the desired value in the same time. The algorithm in this paper, on the other hand, can effectively achieve tracking of trajectories and effectively reduce the error fluctuations to achieve tracking of complex trajectories. Fig.15 shows the input curve in the tracking state of complex curve. In contrast, the inverse control continues to fluctuate and change drastically, and the peak of fluctuation is large. The algorithm in this paper is able to achieve the tracking of complex trajectories after the initial fluctuations while quickly achieving the   convergence of errors, reducing the input peaks as well as reducing the fluctuations. As can be seen from Fig.16, the algorithm proposed in the paper still ensures faster formation control at smaller values of the cost function under complex tracking paths. Fig.17 shows the results of complex trajectory tracking in UAV 3D space under the action of the algorithm in this paper. The pentagram represents the virtual pilot and the circle indicates the follow random. From the trajectory, even in the complex situation, four UAVs can effectively realize the formation and achieve diamond-shaped formation flight. As can be seen from Fig.18, the solution time remains smaller than the sampling time in complex path tracking.

D. MULTI UAVs
To further verify the effectiveness of the algorithm, this section sets up the simulation of the algorithm for formation control of multiple UAVs in complex paths. The initial  conditions of the followers have as well as the desired position settings are shown in Table 4.
And it can be seen from Fig.19 and Fig.20 that the position, velocity and angle errors of the follower UAV can converge to the desired value in a short time, which verifies the fast performance of the algorithm. Meanwhile, the input of the system can converge quickly after the fluctuation in the early stage (see in Fig.21). In Fig.22, the pentagram represents the pilot aircraft, and the diamond and circle of the same color represent different UAVs. From this figure, it can be seen that the algorithm of this paper can effectively achieve the control    of large-scale formation UAVs in complex 3D environment. As we can see in Fig.23, the algorithm's solution time also satisfies the system requirements in a multi-UAV formation.   However, to make the formation converge to the ideal value as soon as possible, the algorithm has a large amplitude for some airframes in the early stages and the solution time also has sampling moments much larger than the average solution time, which will be implemented for improvement in the next paper.

V. CONCLUSION
In this paper, the distributed model predictive control algorithm is proposed for the formation control of formation UAVs in three-dimensional space using relative information. Firstly, a neighboring UAV state estimation model is established, an error model of UAV formation in a 3D environment is established according to the neighboring state estimation model, and a distributed model predictive control algorithm is proposed according to the error estimation model. This paper establishes corresponding cost functions for UAVs in different situations. While giving sufficient conditions for system stability, the stability of UAV formation is proved. The simulation results show that the algorithm can effectively achieve four UAVs' formation control and path tracking under different situations. Finally, to further verify the effectiveness of the algorithm for large-scale UAV cluster control, formation control simulations of eight UAVs in a perturbed environment are conducted, and the results verify the effectiveness of the algorithm. However, when implementing formation control, only the convergence of the formation is considered as soon as possible, resulting in a situation where the input amplitude is large. Future work will focus on improving the algorithm's ability to be used in practice and resist complex perturbations.
QIJIE CHEN received the M.S. degree in control science and engineering from the Naval Aviation University, in 2020, where he is currently pursuing the Ph.D. degree with the Department of Control Science and Engineering. His current research interests include vehicle control and guidance.
YUQIANG JIN received the Ph.D. degree from the Naval Aviation Engineering Institute, in 2009. He is currently an Associate Professor at the School of Operational Services, Naval Aviation University. His current research interests include unmanned systems, multi-intelligent body coordinated control, vehicle control, and guidance.
TAOYU WANG received the M.S. degree in control science and engineering from Naval Aviation University, in 2020. She is currently pursuing the Ph.D. degree with the School of Weapons Science and Technology, Naval Engineering University. Her current research interest includes robust control of servo systems.
YAO WANG received the M.S. degree in military equipment from Naval Aviation University, in 2020, where she is currently pursuing the Ph.D. degree with the Institute of Aircraft Attack, Defense and Operations. Her current research interests include naval weapons and equipment attack, defense, and information technology.
TINGLONG YAN received the M.S. degree in electrical engineering and its automation and the master's degree in engineering in computer technology from Naval Aviation University, in 2018 and 2020, respectively. His current research interests include vehicle control and guidance.
YUFENG LONG received the M.S. degree from the Naval Aviation Engineering School, in 2014, and the Ph.D. degree in control science and engineering, in 2022. His current research interests include UAV troubleshooting and testability studies. VOLUME 10, 2022