Theory and Experiment of Enclosing Control for Second-Order Multi-Agent Systems

This article investigates the enclosing control without preset formation of second-order multi-agent systems for stationary targets. This article uses a directed graph to describe and the direction of information exchange between agents and targets. For continuous-time systems, an enclosing control algorithm is proposed, which does not need to preset the desired formation. The state transfer equation is used to transform the solution of the system into a matrix function of first-order linear constant-coefficient non-homogeneous differential equations. By analyzing the convergence of the solution, the value range of the gain parameter is obtained, and the requirements of topology are proposed. Then the discrete protocol is applied to the discrete-time system. Based on the Schur stability analysis of the system, the requirements of topology and parameter for the system to achieve enclosing control are given. Finally, the self-designed multi-agent platform is introduced, and simulation and experimental results are presented to validate the effectiveness of the protocol.


I. INTRODUCTION
With the development of intelligent robots, cooperative control algorithm has been applied to a great number of civilian and military fields [1]- [4]. According to different application fields, cooperative control can be classified into consensus control [5]- [7], containment control [8]- [10], formation control [11]- [13], consensus tracking [14]- [16], and so on. The relationship between agents in these systems is mainly cooperative. While in some applications, the relationship between multi-agent systems is confrontation. In some application scenarios, such as using robotic shepherds, cluster drone attacks, etc. [17]- [19], agents need to be used to encircle the targets. We refer to the problem of using a multi-agent system to enclose another multi-agent system as enclosing control.
Similar to the enclosing control, containment control involves the encirclement problem of the agents. Containment control can be described as followers designing their control inputs based on information of neighbors to control the agents enter the convex hull formed by the leaders [20]. The containment control of continuous-time systems and discrete-time systems were studied in [21].
The associate editor coordinating the review of this manuscript and approving it for publication was Yilun Shang . Cao and Stuart studied the containment control of secondorder systems under the conditions of fixed topology and switched topology, and realizes containment control on the physical platform [22]. The protocols proposed in these two papers provide a reference for the design of enclosing control protocols. The containment control of first-order and second-order systems was investigated in [23]. The literature divided the containment control problem into two subproblems, namely, the analysis of convergence of system and analysis of topology for ensuring achieve containment control. The analysis of the topology provides a reference for the analysis of achieving enclosing control. Li and Dong studied the problem of formation-containment control, and the platform built in this article reduces the difficulty of control by decoupling the states of quadrotor [24]. It provides a reference for the design of the enclosing control experimental platform.
Although the containment control provides reference analysis methods for enclosing control, there are some differences between containment control and enclosing control. In the containment control, the system controls followers to enter the convex hull formed by the leaders. In contrast, in enclosing control, a multi-agent system is controlled to form a convex hull to contain the targets.
In the existing collaborative control algorithm, formation tracking control and surrounding control can solve some enclosing control problems. The formation tracking control and surrounding control can be described as: the followers calculate their control inputs based on the desire position and set by manual and the information of the neighbors, and control agents to move to the aim position. Kawakami proposed a target encirclement strategy based on virtual structure, which assumed the target as a virtual agent and implemented the encirclement of the target through the consistency algorithm in [25]. In 2014, Aranda studied the encircling of a target with low-velocity characteristics in three-dimensional space [26]. Zhang studied the problem of tracking and surrounding targets using multiple UAVs in [27]. The proposed algorithm controls the alignment of the formation center with the target by keeping multiple UAVs at equal distances, and realizes the tracking and surrounding of the target. One estimator was used to estimate the centroid of multiple agents, and keep the centroid consistent with the target to realize the tracking of the target in [28]. In the research process of formation tracking control of a single target, some scholars began to pay attention to the formation tracking control of multiple targets. Dong proposed a time-varying formation tracking algorithm in [29]. Followers use the information of neighboring agents and relative position information to design the control input, build the formation shape by setting the relative position, and use the consensus algorithm to make the formation track the convex combination of multiple leaders. The work in [30] studied the surrounding control of a second-order multi-agent system with switching topology. Through the estimator, all followers can predict the geometric center of the leaders. By setting the time-varying vector describe desire formation, the followers can realize the circle of the leaders. The surround control problem of an agent with nonlinear dynamic characteristics was studied in [31]. An estimator is designed to estimate the sum of the distance between the leader's geometric center and the leader, and the distance is used for surround control. An adaptive method is given to adjust the position parameters.
When using formation tracking control and surrounding control to solve the problem of enclosing control, it is generally to design vectors manually to describe the desire position between agents, that is, to set a convex hull manually. This process is cumbersome, and mainly used in the enclosing control of a single target. When the formation tracking control is used to solve the problem of enclosing control of multiple targets, it is generally necessary to solve or estimate the set center of the target, so as to transform the problem into the enclosing control of a single target. In some military applications, such as contain multiple targets, agents are limited by the performance of sensors, resulting in the inability to collect information on all targets. The formation tracking strategy is challenging to adapt to such applications. The work in [32] proposes an enclosing control strategy. The strategy uses followers to form a convex hull to contain the leaders by setting the gain parameters and topology only. However, this article only considers the enclosing control of stationary targets of first-order systems. While in many applications, the secondorder integrator is often used to describe the dynamic characteristics of the agents.
Based on the above analysis, this article studies the enclosing control problem of the stationary targets of the second-order system. A continuous-time protocol and a sampled-data based protocol, which is based on position and velocity information, are designed. By analyzing the eigenvalues of the matrix related to the Laplace matrix, the gain range of the continuous-time protocol is obtained. By analyzing the expression of final positions of agents, the requirements of topology are obtained. The Schur stability of the discrete-time system is analyzed, and the requirements of parameters and topology are obtained. Finally, numerical simulation and experimental results are given to validate the theoretical results.
Compared with previous related work and research, the contribution of this article is threefold. Firstly, this article refers to the definition of the convex hull and proposes the relationship between the state vector of agents and that of targets when a multi-agent system encloses the targets. Secondly, the protocols we proposed do not need to design the desire formation, but it is required in [27]- [31]. Thirdly, we discretize the protocol to meet the requirements of engineering applications and propose the requirements of various parameters in engineering practice. Then, the theoretical results are validated by the simulations and an experiment.
The structure of the rest of the paper is as follows: In Section II, some related concepts, definitions, and theoretical results are introduced. The convergences of continuoustime and discrete-time systems are, respectively, analyzed in Section III and Section IV. In Section V and Section VI, the numerical simulations and an experiment are presented, respectively. Finally, Section VII concludes the work of this article.
Notations: ⊗ indicates the Kronecker product. I n is the n-th order identity matrix. diag{α 1 , · · · , α n } represents a diagonal matrix with diagonal elements α 1 , · · · , α n . 0 n×m denotes a zero matrix with n rows and m columns.ẋ represents the differential of x.

II. PRELIMINARY
In this section, we introduce some definitions and theoretical results that need to be used in the following sections.

A. GRAPH THEORY
A directed graph G consists of V, E, and A. Let V be the set of all nodes in the graph that with n nodes. The nodes are denoted by v i , i = 1, · · · , n. E is the set of all edges in graph, which is presented by e ij = (v i , v j ). A is the adjacency matrix of graph, which is represented by a ij , i, j = 1, · · · , n. When e ij ∈ E(G), a ij = 1, otherwise, a ij = 0.
The neighbors and in-degree of node v i are, respectively, defined as: The Laplace matrix of the graph is L = [l ij ], i, j = 1, · · · , n, where when i = j, l ij = −a ij , otherwise, l ij = deg in (v i ). Then, the Laplace matrix of the graph can be obtained by calculating In the research of this article, the existence of repeated edges and self-loops in the communication topology is not considered. However, the theoretical results are still valid for isolated agent nodes.
You can refer to [33] for more detailed content.

B. DEFINITIONS AND LEMMAS
The convex hull of the point set is defined as the minimum convex set of the point set, which is represented by co{v 1 The Kronecker product of matrix has the following properties: 2) When matrices A 1 and A 2 are of the same order, ( When there is a nonsingular matrix P such that B = P −1 AP holds, A is said to be similar to B, and it is denoted as A ∼ B. Similar matrices have the same eigenpolynomial and therefore have equal eigenvalues. Lemma 3 ( [35]): Suppose A ∈ R m×m , B ∈ R n×n , and the eigenvalues of A are λ 1 , λ 2 , · · · , λ m , the eigenvalues of B are µ 1 , µ 2 , · · · , µ n , then the overall eigenvalues of A ⊗ B are λ i µ j (i = 1, 2, · · · , m; j = 1, 2, · · · , n). Lemma 4 ( [35]): For a block matrix, when it is a lower triangular matrix, its eigenvalue is equal to that of the matrix on the diagonal.

C. MATHEMATICAL MODEL OF THE PROBLEM
Suppose that there is a swarm system with n agents and m targets. Let A and T denote the set of agents and targets. Then, the dynamic characteristics of agents and targets can be described as: n are, respectively, the state, velocity and control input of the (i − n)-th agent.
, · · · , n+m are, respectively, the state, velocity and control input of the i-th target. In engineering practice, agents, which number greater than those of targets, are often used to surround the targets. Therefore, we assume that n ≥ m.
In this article, we consider the enclosing control of stationary targets. According to Definition 1, we describe the enclosing control problem as that the agents move according to the position of targets to form a convex hull which contains the targets.

III. CONTINUOUS-TIME PROTOCOL DESIGN AND SYSTEM CONVERGENCE ANALYSIS
We design the following continuous-time control protocol for continuous-time systems: where α is a negative gain constant to be designed. Let According to the protocol, for any agent where n(N i ) denote the number of the neighbors of the i-th agent, n(M i ) is the number of targets for which the agent receives target information. Let L 1 ∈ R n×n denote the Laplace matrix of the multi-agent system. Let D T = diag{n(M 1 ), · · · , n(M i ), · · · , n(M n )}.
Suppose there is a matrix such that L 2 = αL 1 + D T . Construct two matrices as follows: where A T = [a ij ] ∈ R n×m represents the adjacency matrix of the connection between the agents and the targets. When the i-th agent receives the information of the j-th target, a ij = 1, otherwise a ij = 0. Then, according to the definition of the Laplace matrix, the control inputs of the multi-agent system and targets can be written as: According to (4), we can obtain that Substitute (6) into (5) to get a simplified formula: where v T (t) = 0 1×Nm . From Lemma 1, we can get that the solution of the first-order linear constant-coefficient nonhomogeneous differential equations of the matrix function are: Assuming E is a non-singular matrix, The following theorem shows the range of parameter and the requirements of topology to achieve enclosing control.
Theorem 1: Suppose the topology is fixed. The convex hull formed by the agents contains the targets for any initial state if the control gain such that max −1 2(L 1 ) ii < α < 0, where (L 1 ) ii is the element on the diagonal of the i-th row of L 1 , and each agent receives information from one target only, and there is no isolated target.
Proof: When the eigenvalues of the E all have negative real part, Then the solution of (7) is: Construct a matrix P such that P −1 EP = E 1 , where Then, where (L 1 ) ii , (L 2 ) ii are, respectively, the element on diagonal of the i-th row of L 1 and L 2 . Since each agent receives information from one target only, by calculation, the requirements of the range of α can be obtained as following.
Since each agent receives information from one target only, D T = I n . According to Lemma 6, we can get expressions of E −1 by calculation.
Substituting (16) into (8), we can get that When t → ∞, we can obtain that

VOLUME 8, 2020
Since each agent receives information from one target only, and there is no isolated target, A T is full column rank. According to the expression of L 2 , the state of i-th target can be expressed as: where = [−αa i1 , · · · , αn(N i ) + n(M i ), · · · , −αa in ], i = 1, · · · , n. According to the range of α, −αa ij > 0, i, j = 1, · · · , n, i = j. From (14) and the expression of L 2 , we can obtain that αn(N i ) + n(M i ) = (L 2 ) ii > 0. From the expression of , we can get the sum of the coefficients of is According to Definition 1, when t → ∞, the convex full formed by agents contains the targets.
Remark 1: When α ≤ max −1 2(L 1 ) ii , the positions of agents tend to infinity, and the convex hull formed by agents has no application significance. When the α increases within the value range, the area enclosed by the convex decreases as the alpha increases. When α = 0, the convex hull formed by the agents is consistent with the convex hull formed by the target.

IV. SAMPLED-DATA BASED PROTOCOL DESIGN AND SYSTEM CONVERGENCE ANALYSIS
We use constant time interval sampling and zero-order hold circuit to discretize the protocol (2) to get the following protocol.
From Newton's second law, we can obtain: Then, the state transition equation can be obtained as: Then according to the state transition equation, the state at the kh + h time of the system is: Then the conditions that ensure achieve enclosing control are proposed as following.
Theorem 2: Suppose the topology is fixed. The convex hull formed by the agents contain the target for any initial state if the parameters such that max −1 2(L 1 ) ii < α < 0, h < 2 λ , and h < 2, where λ presents the denote the eigenvalues of L 2 , and each agent receives information from one target only, and there is no isolated target. The state of agents can be obtained by solving (L 2 Proof: When the eigenvalues of the 1 are all within the unit circle, lim k→∞ is eigen-matrix of L 2 . Let λ 1 , . . . , λ n denote the eigenvalues of L 2 . Then (L 2 ) = diag{λ 1 , λ 2 , . . . , λ n }. Since each agent receives information form one target only, D T = I n . Therefore, The eigenvalues of 1 are equal to those of W −1 1 W .
The eigenvalues of 1 can be obtained by let a i (s) = 0, i = 1, · · · , n. Let λ denote all the eigenvalues of L 2 . By some calculation, we can get that When a(s) is Schur stable, the eigenvalues of the 1 are all within the unit circle. According to Lemma 7, using bilinear transformation σ +1 σ −1 , we can obtain that The Schur stability analysis of a(s) is transformed into the Hurwitz stability analysis of r(σ ). According to Lemma 8, when all the coefficients of r(σ ) are bigger than zero, r(σ ) is Hurwitz stable.
From the analysis in the Section III, when max −1 2(L 1 ) ii < α < 0, all the eigenvalues of L 2 are bigger than zero. Therefore, the requirement of range of parameters such that Let lim k→∞ k 1 = 0 2n×2n , by some calculation, we can obtain that Substituting (32) into (24), we obtain that Then From (34), we can get that Similar to the analysis of expression of x n (t) in Section III, we can obtain the conclusion that when t → ∞, the convex hull formed by agents contains the target.
Remark 2: The protocol is discretized so that it can be deployed in engineering applications. In our theoretical analysis, the existence of external interference is not considered, and the maximum velocity of the agents is not limited. In actual engineering practice, the speed of the agents has an upper limit. When the external interference input is greater than the maximum control input, the protocol will fail.

V. SIMULATIONS
We use two examples to validate the theorem result in this section.
We regard an agent with the dynamic of the second-order integrator as the target and use four agents to form convex full, which encloses the targets. The direction of information exchange between the targets and the multi-agent system is shown in Fig. 1, where T i , i = 1, 2, 3, are the targets, and A i , i = 1, 2, 3, 4, are agents. From Fig. 1, we can get the expression of L 1 and A T as following: According to Theorem 1 and Theorem 2 we can obtain that −0.25 < α < 0. Example 1: Use (2) for (1). According to Theorem 1, we can obtain that −0.25 < α < 0. Let α = −0.2, and simulation time is 40 seconds. The trajectories of agents are shown in Fig. 2. Obviously, the convex hull formed by the agents contains the targets. The velocity curves of the agents in the coordinate axis directions are shown in Fig. 3. The simulation results validate Theorem 1.
Then we can obtain that h < 2. Let α = −0.2, h = 1.2, the simulation time is 40 seconds. Obtain the trajectories of the agents as shown in Fig. 4. We can see that the convex full formed by agents contains the targets. The velocity curves of the agents are shown in Fig. 5. The simulation results validate Theorem 2.
Some strategies, such as formation tracking [25]- [29] and surround control [30], [31], can solve some enclosing control problem, but it is generally to manually set a vector describing the desired formation to control followers to form a desire formation to surround the leaders (targets). However, the enclosing control strategy proposed in this article only needs to set the relevant parameters and does not need to preset the desire formation so that the agents can enclose the targets.

VI. EXPERIMENT
In this section, We introduce the self-designed multi-agent platform and the experiment with the multi-agent platform. Next, this section introduces the hardware structure of the multi-agent platform. The platform consists of Ultra Wide Band (UWB) positioning system, azimuth correction system (ACS), five omnidirectional wheel robots, and a control station. As shown in Fig. 6, the UWB positioning system with an accuracy of 5cm circular error probable consists of tags with a positioning function and base stations that provide a positioning reference. The positioning system uploads the position of each robot to the control station through the first base station.
When there is an angle between the robot coordinate system and the world coordinate system, the control input in one direction will cause the robot to move in the other direction. It is necessary to make the angle between the robot coordinate system and the world coordinate system approach zero. Therefore, we designed the ACS for the platform.
The ACS consists of ACS circuit and the Repeater. The ACS circuit collects the azimuth information of the robot and uses the nRF24L01 (NRF) module to upload the azimuth to the control station. After receiving the azimuth angle, the control station makes a difference with the azimuth information of the world coordinate system to obtain the angle between the robot coordinate system and the world coordinate system. The control station sends a correction command to make the robot correct the included angle.
The software of the multi-agent platform includes the top and bottom parts. The bottom part of the software is mainly programmed using Embedded C and is used to control robots, ACS circuits, and repeaters. The top part of the software is developed in the Windows10 system using python. The main functions include: i) designing the topology through programming; ii) receiving positioning and azimuth information; iii) calculating the correction input and control input of each robot; iv) sending control information for control robots every 0.2 seconds.
We use three robots to simulate the targets and four robots to mimic the agents. Through programming, the communication topology of the agent robots and the simulated targets are set to the structure, which is shown in Fig. 1. Similar to Example 2 in Section V, −0.25 < α < 0. Let α = −0.2, then we can know h should satisfy h < 2. The control period of the platform is smaller 2 seconds. Let the experiment time is 40 seconds. Draw trajectories of the location of the agents to get the Fig. 7.
Since we did not limit the maximum speed of the agent in the simulation, and due to the existence of positioning error and angle between the robot coordinate system and the world coordinates, there are some errors between the trajectory of the experiment and that in the simulation. Therefore, there is an error between the final position of agents in the experiment and that of numerical simulation. However, it can be seen from the trajectories in Fig. 7 that the target robots are enclosed by the agent robots. Moreover, it can be seen from Fig. 8 that the velocity of the agent approaching zero as time increases. These results validate Theorem 2.
Then, we consider the boundary conditions assumed in Subsection C of Section II, let n = m, and use the communication topology, as shown in Fig. 9, for simulation and experiment. Set the simulation time and experiment time as 40s, h = 0.2, α = −0.2. The simulation of the sampleddata based protocol and experimental results are shown in Fig. 10. It can be concluded from Fig. 10 that the experimental results are consistent with the simulation results when the positioning and control errors are ignored.  It is worth noting that only one undirected graph is used for simulation and experiment. Still, the theoretical results of VOLUME 8, 2020 this article are also applicable to multi-agent systems with directed communication topology.

VII. CONCLUSION
This article has studied the enclosing control problem of stationary targets. First, the second-order integrator was used to describe the dynamics of the agents and targets, and the directed graph was used to present the direction of information exchange between the multi-agent system and the targets. Then, we proposed a continuous-time protocol and sampleddata based protocol. In continuous-time systems, the problem of solving the final state of the agents was transformed into the problem of solving matrix function equations. By analyzing the eigenvalues of the matrix related to the Laplace matrix, the parameter range was derived. The requirements of topology were obtained by analyzing the expression of the final positions of the agents. In discrete-time systems, by analyzing the Schur stability of the system, the range of the parameter of ensuring the system to achieve enclosing control was obtained. Finally, two numerical simulations and one experiment were used to validate the theoretical results.
In this article, the closed control of second-order multiagent is studied, and some factors are ignored in the research. Some interesting directions for future research are listed as follows.
1) In practical engineering applications, the topology of multi-agents may be time-varying, so the enclosing control of multi-agent systems with deterministic switching or random switching topology should be studied.
2) The enclosing control for dynamic targets is a problem worthy of attention.
3) The parameter setting of the protocols proposed in this article is a problem worth investigating.