Adaptive Neural Network Leader-Follower Formation Control for a Class of Second-Order Nonlinear Multi-Agent Systems With Unknown Dynamics

In this article, an adaptive leader-follower formation control on the basis of neural network (NN) is developed for a class of second-order nonlinear multi-agent systems with unknown dynamics. Unlike the first-order formation control that only needs to govern the position states, the second-order formation control needs to govern both the position and velocity variables. Hence the second-order formation is more challenging and interesting than the first-order case. In the control design, the adaptive NN approximator is employed to compensate the nonlinear uncertainties, so that the control design difficulty coming from the unknown dynamics is effectively overcome. Through Lyapunov stability analysis, it is demonstrated that the proposed control method can complete the control tasks. To further demonstrate the effectiveness, the formation method is implemented to a numerical simulation, and it shows the desired results.


I. INTRODUCTION
Since the multi-agent control theory had been significantly developed, and lots of research results were published in literatures. In recent decades, their applications are increasingly being implemented to various fields, such as clusters of satellites, sensor networks, robot team, coordination of unmanned air vehicles [1]- [4]. A multi-agent system is made up of multiple intelligent individuals under interaction. Because of the outstanding performances, for examples, flexibility, reliability, efficiency and new capabilities, the multi-agent can finish many difficult and arduous tasks, and it can surpass the ability of multiple single agents. Therefore, multi-agent control is meeting the requirement for developing modern military, modern industry and modern civilian.
Formation control is to find the control protocol for a multi-agent system so that the system states are driven to arrive the prescribed constraints finally. Due to wide The associate editor coordinating the review of this manuscript and approving it for publication was Ning Sun . applications, formation control is attracting the increasing attention coming from the various areas. In general, several basic formation control strategies are leader-follower [5], behavior-based [6], virtual structure [7] and potential function [8]. Particularly, leader-follower formation strategy is the most popular one owing to the simplicity and scalability. Recently, increasing attention is being devoted to the neighbor-based formation control protocol [9], of which the main advantage is that formation controller only needs the information of a small number of agents.
Since the first-order multi-agent formation control just needs to govern the position state, the demonstration of technical feasibility is with simpler mathematics. After several decades of development, lots of research results were reported in literatures [10]- [15]. However, unlike the first-order multi-agent formation, the second-order case needs to govern both the position and velocity variables. So the second-order multi-agent formation is more interesting and challenging than the first-order case. In past decades, the second-order multi-agent formation control received VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ considerable attention, many gratifying research results have been reported in the literatures [16]- [19]. It's a fact that the unknown nonlinear dynamic is ubiquitous in the real-world engineering systems. However, most of the published nonlinear multi-agent control methods either assume one of both the known dynamic function and the Lipschitz continuous condition, or have many mathematic imperfect in the theory proof, for examples [20]- [23]. Since neural networks (NNs) are proven to have universal approximating and adaptive learning abilities, they are combined with a lot of control techniques to deal with the unknown dynamic problem of nonlinear systems, such as backstepping, observer, dynamic surface, H ∞ technique [24]- [28]. Recently, several adaptive nonlinear multi-agent consensus control methods using NN are reported [29]- [32], they further demonstrate that NN can be an effective tool to solve the unknown dynamic problem for nonlinear multi-agent systems. Moreover, fuzzy logic systems (FLSs) are also demonstrated to have excellent approximation ability, and have been well applied to adaptive nonlinear control, such as [33]- [36]. Unfortunately, to the second-order multi-agent formation, NN or FLS adaptive control are rarely reported.
Motivated by the above analysis, this paper develops an adaptive formation control method for a class of nonlinear double integral unknown dynamic multi-agent systems. A numerical simulation example is performed via the proposed control method, and it also shows the desired results. The main contributions are listed in the following.
1) A formation control scheme for the second-order nonlinear multi-agent system is developed. Based on the Lyapunov stability theorem, it is proven that the control scheme can ensure to achieve the desired control objective. 2) The unknown dynamic problem is solved by employing the adaptive NN. The NN in the control is used to approximate the nonlinear dynamic function, so that the unknown dynamic is effectively compensated. The paper is organized as follows. In section 2, the preliminaries concluded neural networks, graph theory, and supporting lemmas are introduced. In section 3, the main results involved problem formulation, control protocol design, and theory proof are presented. In section 4, a simulation example is provided. Section 5 is the conclusion.

A. NEURAL NETWORKS
Neural networks (NNs) has been demonstrated to have the universal function approximation ability. Give a continuous function f (x) : R m → R n , NN can approximate the function over a compact set in the following form: where W ∈ R q×n is the weight matrix with the neuron number q, S(x) = s 1 (x), · · · , s q (x) T is the basis function vector For the continuous function f (x), there exists the ideal NN weight W * ∈ R q×n described as so that f (x) can be rewritten as where ε(x) ∈ R n is the approximation error, and there exists a positive constant δ such that ε(x) ≤ δ. The ideal NN weight W * is to ensure the minimum possible deviation between W T S(x) and f (x). However, it cannot be directly applied to design the control protocol because it is just an ''artificial'' quantity for analyzing. Usually, its estimation obtained via adaptive tuning is used to construct the actual control.

B. GRAPH THEORY
For the multi-agent system in this paper, the interconnected graph is assumed to be an undirected connected graph G = (A, M , ), where A = a ij ∈ R m×m is the adjacency matrix whose the element a ij ≥ 0 is the communicated weight between agents i and j, M = {1, 2, · · · , m} is the label set of all nodes, ∈ M × M is the edge set. If the edge ξ ij holds ξ ij = (i, j) ∈ , then it means exiting a communication from node j to node i, and the node j is said to be a neighbor of the node i, and i = {j|(i, j) ∈ } denotes the neighbor label set, and the adjacency element a ij associated with the edge ξ ij is assigned a ij = 1. If ξ ij / ∈ , then a ij = 0. If the adjacency elements of matrix A is with the property a ij = a ji , i, j = 1, · · · , m, which means ξ ij ∈ ⇐⇒ ξ ji ∈ , then the graph G is called as an undirected graph. The undirected graph G is called to be connected if there is an undirected path, (i, i 1 ), · · · , (i i , j), for any two different nodes i and j. Associated with the graph G, the Laplacian matrix is The communication weights between agents and leader are described by the matrix D = diag {d 1 , · · · , d m }. If the agent i can have the communication with the leader, then d i = 1; otherwise d i = 0. It is supposed that d 1 + · · · + d m ≥ 1, which implies that at least one of agents connected with the leader.
C. SUPPORTING LEMMAS Lemma 1: [37] If an undirected graph G is connected, a necessary and sufficient condition is that its Laplacian matrix is irreducible.
where d 1 , · · · , d m are nonnegative constants satisfied d 1 + · · · + d m > 0. Lemma 3: [21] The matrix inequality that where , is equivalent to any one of the following two inequalities: i) where a and c are two positive constants, then the following inequality can be held

A. PROBLEM FORMULATION
Consider the nonlinear multi-agent system that is made up of m agents modeled by the following double integral dynamics: where The desired reference signals are described by the following dynamics, which is viewed as an independent leader agent,ẋ (t) =ȳ(t) wherex ∈ R n is the reference position,ȳ ∈ R n is the reference velocity, h(·) ∈ R n is a smooth bounded function [11], [24]. Definition 1 ( [18]): The second-order leader-follower formation is achieved if the solutions of multi-agent system (7) satisfy lim is a constant vector, which describes the desired relative position between agent i and the reference (8).
The Control Objective: Design an adaptive formation control protocol for the nonlinear multi-agent system (7) so that i) all error signals are semi-globally uniformly ultimately bounded (SGUUB); ii) the second-order leader-follower formation is achieved.

B. CONTROL PROTOCOL DESIGN
Define the following coordinate transformations as According to (7) and (8), the following error dynamics are yielded asż Rewrite the error dynamic (10) to the compact form aṡ where Define the position and velocity formation errors as where a ij and d i are the elements of the matrices A and D, which are introduced in Subsection II.B, i is the neighbor label set of agent i. Furthermore, the two formation error terms in (12) can be rewritten on the basis of (9) as In (11), the nonlinear function f i (x i , y i ) is unknown but continuous, give a compact set i ⊂ R 2n , for [x T i , y T i ] T ∈ i , it can be re-described by using its NN approximation as where W * i ∈ R q i ×n is the ideal NN weight matrix with the NN neuron number In (14), since the ideal weight matrix W * i is an unknown constant matrix, it is unavailable for the actual control design. By using the estimationŴ i (t) of the ideal NN weight W * i , the formation control is constructed in the following: (15) where γ x > 0, γ y > 0 are two design constants,Ŵ i (t) ∈ R q i ×n is the estimation of W * i .

VOLUME 8, 2020
The NN updating law for tuningŴ i (t) is given in the following: (16) where i ∈ R q i ×q i is a positive definite matrix, σ i > 0 is a design constant.
Remark 1: In the controller (15), both the position error term e xi (t) and velocity error term e yi (t) defined in (12) aim to control the multi-agent system arriving and keeping the formation pattern and velocity consensus, respectively. The NN termŴ T i (t)S i (x i , y i ) aims to compensate the unknown dynamic via on-line tuning the NN weightŴ i (t) using the updating law (16). Since the proposed scheme provides a basic formation control ideal for the second-order nonlinear multi-agent systems, it can be applied and extended by combining with various control techniques, such as H ∞ robust control [28], reinforcement learning [39].

C. THEOREM WITH PROOF
Theorem 1: Consider the nonlinear second-order multiagent system (7) with the bounded initial conditions under the undirected connected graph G. If the adaptive formation control (15) with the NN weight updating law (16) is performed for the multi-agent system, and the design constants are chosen to satisfy where λL min is the minimal eigenvalue of matrixL, then the following control objectives can be achieved. 1) All errors are SGUUB.
2) The multi-agent formation can be achieved for sufficiently smooth movement trajectory.

Proof: The Lyapunov function candidate is chosen as
According to Lemma 2, it can be directly concluded that the symmetric matrixL is a positive definite matrix. If the design parameters satisfy the condition (17), there is the fact (γ x + γ y )LL −L > 0. Hence, (γ x + γ y )LLL LL is a positive definite matrix in accordance with Lemma 3. And thus the function V (t) can be considered as a Lypunov function candidate.
The time derivative of V (t) along (11) and (16) iṡ Using the facts e x (t) =Lz x (t) and e y (t) =Lz y (t), where e x (t) = [e T x1 (t), · · · , e T xm (t)] T ∈ R mn and e y (t) = [e T y1 (t), · · · , e T ym (t)] T ∈ R mn , the equation (19) can be rewritten aṡ After several simple operations, the following one can be derived from (20), Inserting the NN approximation (14) and the controller (15) into (21) yieldṡ In the light of the equationW i (t) =Ŵ i (t) − W * i , the equation (22) can be rewritten aṡ According to the property of trace operation, Using the above equation (24), the equation (22) can become the following onė Applying Cauchy-Buniakowsky-Schwarz inequality and Young's inequality, the following results can be get e T yi (t)z yi (t) Substituting the inequalities (26) into (25) yieldṡ UsingW i (t) =Ŵ i (t) − W * i , the following equation can be obtained Substituting (28) into (27) haṡ Applying Lemma 4 to (31), there is the following inequality From the above inequality, it can be proven that i) these errors z xi (t), z yi (t),W i (t), i = 1, · · · , n, are SGUUB; ii) the tracking errors z xi (t), z yi (t), i = 1, · · · , n, can obtain the desired accuracy by choosing the design parameters large enough, and it means that the multi-agent formation can be achieved.

IV. SIMULATION
In this simulation example, the multi-agent system moving on the 2-D plane is composed of 4 agents, and it is described in the following: where In accordance with the control conditions (17), the formation control with respect to (15) chooses the design parameters γ x = 60 and γ y = 40. The NN is designed to have  12 neurons, and the centers µ i are also evenly spaced from −6 to 6. The updating law with respect to (16) Fig.1 shows that the desired multi-agent formation is achieved, and Figs.2-3 show the velocity tracking performance. Fig. 4 shows the NN weights to be bounded. These simulation figures show that the desired results are obtained, it can be concluded that the proposed formation method can achieve the desired control tasks and objectives.

V. CONCLUSION
In this paper, a leader-follower adaptive NN formation control is developed for a class of second-order multi-agent systems under unknown nonlinear dynamics. In the proposed control scheme, NN is used to approximate the unknown dynamic functions, then, based on the NN approximation, the proposed control scheme is constructed to solve the unknown dynamic problem. According to Lyapunov stability analysis, it is proven that the proposed adaptive NN formation method can realize the control objectives. Simulation results also show the desired control performance.
To the best of authors' knowledge, most of multi-agent optimal control is focused on the first-order case. Therefore, our future works will consider optimal control of second-order nonlinear multi-agent systems. Based on the proposed control scheme, by employing reinforcement learn, we will develop optimal control for nonlinear second-order nonlinear multi-agent systems. YANG CUI received the B.S. degree in information and computing science and the M.S. degree in applied mathematics from the Liaoning University of Technology, Jinzhou, China, in 2009 and 2012, respectively, and the Ph.D. degree in control theory and control engineering from Northeastern University, Shenyang, China, in 2018. She is currently a Lecturer with the School of Electronic and Information Engineering, University of Science and Technology Liaoning, Anshan, China. Her research interests include dynamic surface control, adaptive control, nonlinear control, and multiagent control.