A Consensus Control for a Multi-Agent System With Unknown Time-Varying Communication Delays

A multi-agent system to have information flow among agents is bound to experience a communication delay over a directed graph. However, delay information is often unavailable and time varying. To deal with this problem, a consensus control for a second-order multi-agent system consisting of a leader and multiple followers is developed. Sliding mode control is exploited to deal with uncertainties incurred by unknown time varying communication delay and disturbance. The proposed sliding mode consensus control is shown to achieve the asymptotic bounded consensus and the finite time convergence of a sliding variable for any unknown time varying delay with the bounded first and the second derivative. The numerical simulations verify the properties of the proposed algorithm and its performance which is almost identical to one achieved by a consensus algorithm exploiting known delay information.


I. INTRODUCTION
With the widespread use of the internet of things, connected devices which are working at geographically different areas often work as a single complex system. A multi-agent system (MAS) in which each agent works independently or dependently to achieve a shared goal also consists of physical or virtual agents [1]. The MAS may be controlled from a single controller with very powerful processing capability, which is called as a centralized control. However, this control strategy may have limited applicability due to the complexity which is proportional to the number of agents. On the other hand, decentralized control allows each agent to operate independently without sharing information, which may often incur the performance degradation compared to the centralized control. Alternatively, a distributed control which exploits information from the limited number of agents often provides the good tradeoff between the centralized control and the decentralized one [2].
A consensus control as one of the distributed control algorithms tries to align the state or dynamics of the agents The associate editor coordinating the review of this manuscript and approving it for publication was Shihong Ding .
in a coordinated way [3]. The consensus control has been exploited to many practical problems such as coordination of multiple aircrafts [4], formation control with multiple wheeled robots [5], and demand side management for smart grid [6]. A consensus protocol is known to originate from management science [7]. Significant research on consensus control for a MAS has been done since early pioneering researches in the theoretical foundation based on graph theory were done [7]- [9]. A linear consensus protocol with Laplacian matrix was shown to be the optimal linear quadratic control for a MAS with the first-order dynamics [10]. To save network bandwidth and computing resources, an event-based consensus control was developed [11]. Combining Kalman filter with a consensus protocol, a distributed consensus estimation was also studied [12].
The implementation of consensus control necessitates information flow among agents, which accompanies a communication delay. In addition, the structure of a system or computational complexity may also incur a system delay. The development of a consensus control for a MAS with a delay can be broadly classified into two types, time-domain approach and frequency domain approach. The time-domain approach often exploits the Lyapunov stability criterion or predictive control to derive a consensus condition. A consensus condition in the presence of random link failure was derived from the Lyapunov stability condition [13]. A sufficient condition for the average consensus for a MAS with higher order dynamics and multiple time varying bounded delays was given as linear matrix inequalities (LMIs) from exploiting the free weight matrix method [14]. A consensus control was designed with prediction to compensate for the delay in a linear system with input delays and output delays [15]. An output consensus protocol for a switched heterogeneous MAS with a communication delay was developed to be resilient to uncertainties from asynchronous switching and communication delays [16]. An asynchronous event-triggered consensus protocol for a MAS with double integrator dynamics in the presence of the input delay was shown to achieve an average consensus for the delay satisfying a bound condition [17]. Many frequency domain approaches apply Laplace transform to the consensus equation and apply the Nyquist criterion to derive consensus conditions. A consensus control for a heterogeneous MAS with the first-order dynamics and the second-order dynamics was proposed to achieve consensus for the communication delay when the input delay satisfied a bound condition [18] or the input delay satisfied some conditions [19]. Group consensus condition for a heterogeneous MAS with the first-order dynamics and the second-order dynamics was given as a function of topology and input delay [20] while a consensus protocol designed with cooperative and competitive interactions was shown to achieve the stationary position consensus regardless of communication delays [21].
Despite the significant number of researches on consensus control for a MAS with delay, there are few existing literatures on research for a MAS with unknown fixed delays or time-varying delays. A consensus control for a MAS with higher-order dynamics and unknown fixed delays was proposed with an adaptive delay estimation algorithm to satisfy the match condition rather than to estimate each delay separately [22]. A robust consensus control with sliding mode for a nonlinear MAS achieved a stationary consensus in the presence of bounded system uncertainty and unknown fixed communication delays [23]. An adaptive consensus protocol was shown to achieve the controllable leader-follower consensus in the presence of unknown fixed delay and nonlinear uncertainty by estimating the nonlinear uncertainty with the neural network [24]. A leader-follower consensus protocol for a MAS with partial mixed impulses and unknown same time-varying state delay was proven to achieve exponential consensus under a specific condition [25]. A simple consensus protocol for a second-order leader-follower MAS with unknown same time-varying input delay was shown to achieve consensus as long as the maximum delay is less than a threshold determined by a consensus condition [26]. A delay partitioning method and Wirtinger-based inequality were exploited to derive a second order consensus control for a MAS with unknown time varying input delay, of which sufficient condition for an asymptotic consensus was less conservative [27]. A group consensus protocol for a MAS with unknown fixed communication delay was shown to achieve stationary consensus [28]. An average consensus condition for a consensus protocol with time-varying delay was derived as a set of LMIs which were the functions of bounded delays and the bounded first-order time variations from exploiting Lyapunov Krasovskii (LK) functional [14]. With a similar procedure, the LMI condition for the consensus of a MAS with identical time varying system delay only was derived [29]. A consensus control for a heterogeneous MAS with unknown time varying system delays and unknown bounded nonlinear dynamics achieved the consensus on the dynamics of the leader node with bounded asymptotic error [30]. A sufficient condition for the consensus of a MAS with a Markov delay was given as LMIs from transforming the consensus problem into the stabilization of error dynamics [31].
Delay information is often unavailable and time-varying. Despite its practical importance, to the best of the author's knowledge, there is no research on consensus control for a MAS with unknown time varying communication delays properly. Existing researches assume unknown fixed delays [23], [28], known time-varying communication delay [29], [31], or unknown time-varying state delay [25], [26]. In addition, every physical system is bound to have uncertainties from system structure or disturbances. To deal with these uncertainties together, a sliding mode consensus control for a second-order MAS with unknown time-varying communication delays and disturbance is developed. The proposed consensus control is shown to achieve the finite consensus error which is the difference between the state of each agent and that of the leader agent as long as the state of each agent changes finitely over the communication delay while the finite time convergence of a sliding variable is guaranteed. To this end, a consensus control for a MAS without delays to achieve perfect consensus in the presence of disturbance is proposed. Then, a consensus control for a system with known delays and disturbances is developed with modifying one for the system without delays. The disagreement vector of the consensus control is shown to be bounded by a finite value. Finally the consensus control for a system with unknown time varying delays and disturbances is developed. This paper is organized as follows. In section-2, the consensus problem for a MAS is defined with graph theory, and associated assumptions. A sliding mode consensus control for a MAS without delays is proposed in Section-3. The finite time convergence of an associated sliding variable and an asymptotic perfect consensus are proved. In section-4, a sliding mode consensus control for a MAS with disturbance and unknown delays is developed. The theoretical properties of the proposed consensus controls are verified with a simple hypothetical MAS in section-5. Some concluding remarks and future research directions are made in section-6.
The following notations are used throughout this paper. diag(a 1 , a 2 , · · · , a N ) is a diagonal matrix of which the ith element is a i . [x] i denotes the ith element of the vector x while VOLUME 9, 2021 [X ] i does the ith row of the matrix X . The vector of length N with all 1 elements is denoted as 1 N . The time index of a continuous signal will be omitted for simplicity of notation unless otherwise required for clarity.

II. A SYSTEM MODEL AND PROBLEM FORMULATION
We consider a MAS with one leader and N followers where every agent follows the double integrator dynamics. This model has been often exploited to the control of agents which move in a coordinated way [17], [32], [33]. The ith agent can be given asẍ is a control signal and v i (t) is disturbance or system uncertainty. The index of the agent corresponding to the leader agent is assumed to be 0. The communication network for N follower agents can be presented by a graph where V N is a set of nodes representing each agent, and Q N is a set of edges representing the information flow among nodes. Let q ij be the directed edge from the node j to the node i. The adjacency matrix A N which represents the information flow can be defined as where a ij is the element in the ith row and the jth column of the matrix A N . The Laplacian matrix L N can be defined from the adjacency matrix as where l ij is the element in the ith row and the jth column of the matrix L N . It is well known that L N always has the single eigenvalue of 0 with the eigenvector 1 N which is a column vector of all ones when L N has a rooted directed spanning tree.
To define the problem clearly, assumptions and definitions are made as follows.
Assumption 1: A graph formed by the communication network of the N followers has a rooted directed spanning tree.
Assumption 2: At least one follower agent receives information from the leader.
Assumption 3: The magnitude of disturbance is bounded by a finite value which is known.

Definition 2 (Asymptotic Bounded Consensus of the Leader-Follower MAS):
A MAS with one leader and N followers is said to achieve asymptotic bounded consensus on the position and velocity if the following conditions are satisfied.
where t 0 , δ p and δ v are some positive finite constant. The goal of this paper is to find a consensus control to achieve the asymptotic bounded consensus of the leader-follower MAS in the presence of disturbance and unknown time-varying delays.

III. A SLIDING MODE CONTROL FOR A MULTI-AGENT SYSTEM WITHOUT COMMUNICATION DELAY
A sliding mode control is known to provide robust control in the presence of uncertainties. In this section, The sliding mode consensus protocol for a leader-follower MAS without communication delays will be developed so that it can be extended to the system with unknown time-varying communication delay. We first define a topology dependent disagreement vector of which the ith element is written as With e i , we propose a sliding variable for the ith agent as where a i,sum = N j=0 a ij . The corresponding dynamics in the sliding surface is given bẏ The stability condition of the topology dependent disagreement dynamics is given in the following proposition. Proposition 1: As long as c > 0, the topology dependent disagreement dynamics in the proposed sliding surface is asymptotically stable.
Proof:ẋ i can be rearranged from (6) aṡ After inserting (9) into (8), (8) can be expressed aṡ Since a i,sum is greater than 0, due to the assumption-1, any positive c guarantees the stability of the topology dependent disagreement dynamics from Lyapunov stability condition.
To induce the dynamics in the sliding surface, the sliding variable s i needs converge in a finite time. This can be achieved by designing a control u i , in the following way.
Theorem 3.1: Let the proposed consensus control u i be given by Then, s i converges to 0 in a finite time.
Proof: Let a positive definite Lyapunov function V i be given by 0.5s 2 i . From (1) and (7),ṡ i can be expressed aṡ (12) where u i = u e,i + u s,i . By setting u e,i as cė i + a −1 a ijẍj ),ṡ i can be rearranged aṡ Let u s,i be given by −k u sign(s i (t)). We can evaluateV i with three different cases. When s i > 0, (14) follows from the condition k u > max i,t (|v i (t)|) which is given in the theorem. When s i < 0 Finally, when s i = 0, it already converged to 0.V i < 0 proves that s i converges to 0 asymptotically. To prove convergence in a finite time, Let k u − max i,t (|v i (t)|) be denoted by α u . When After proving thatV i < α u s i when s i < 0 with the same procedure, we haveV which proves the finite convergence. It is noted that the consensus control in (11) requires local information onẍ j from the neighbor agent only while it can count the number of directed edges from neighbor agents right away. In addition, the convergence speed is found to be proportional to the magnitude of k u from (17).

IV. A SLIDING MODE CONTROL FOR A MULTI-AGENT SYSTEM WITH COMMUNICATION DELAYS
In this section, two sliding mode consensus protocols will be developed for a MAS with communication delays. The MAS may be able to estimate the communication delay when all agents are connected on a single network or time stamp is provided on a data packet. However, when they are connected on complex heterogeneous networks or the time stamps are not available on the data packet, accurate delay estimation may not be feasible due to time varying transmission delay due to time varying nature of the delay. Thus the sliding mode consensus protocol will be developed with delay information and without delay information respectively.

A. A SLIDING MODE CONSENSUS CONTROL WITH KNOWN DELAYS
Similarly to the case without delay, a topology dependent disagreement vector of which the ith element for a MAS with known delay can be defined as where τ ij (t) is a communication delay from the agent j to the agent i. We can also define a sliding variable in the following way such that e i can converge to zero asymptotically in a sliding surface.
The topology dependent disagreement vector will be stable as long as c > 0. To induce the dynamics in the sliding surface, the sliding variable s i needs to converge in a finite time. This can be achieved by designing a control u i in the following way. Theorem 4.1: Let the proposed consensus control u i be given by where k u > max i,t (|v i (t)|). Then, s i converges to 0 in a finite time.
Proof: Let a positive definite Lyapunov function V i be given by 0.5s 2 i . From (1), (18), and (19),ṡ i can be expressed asṡ where u i = u e,i + u s,i . By setting u e,i as a −1 i,sum (−cė i + p i ),ṡ i can be rearranged asṡ i = u s,i + v i . Setting u s,i by −k u sign(s i ) results inV i < 0 with the same arguments made in the proof of the theorem-3.1. The proof of the convergence in finite time can be also proved by following the same arguments made in the theorem-3.1.

VOLUME 9, 2021
It is noted that in addition toẍ j from the neighbor agents and a i,sum , the consensus control in (20) requires information on τ ij (t),τ ij (t), andẋ j . Thus, information transmission from the neighbor agents will be doubled in comparison to the case without delays. Even though the theorem-4.1 guarantees that the proposed consensus control can make e i converge to 0, it does not mean that it achieves the asymptotic perfect consensus of the leader-follower MAS due to the delay terms in (18). When every e i converges to 0, the following vector equation can be constructed from (18). where and e x,0 andẽ x,i are defined respectively as The disagreement vector q x resulting from the proposed consensus control can be given from (22) as It is noted that the disagreement vector depends on two terms, the degree of the leader variation over the delay time, and the degree of follower variation over the delay time. The inverse of (L N + D 0 ) also implies that the magnitude of the disagreement vector is likely to be larger with a sparse graph than with a dense graph. The properties of the proposed consensus can be summarized from (24) by the following proposition. Proposition 2: The consensus control given in (20) for a MAS with known communication delays achieves the asymptotic bounded consensus of the leader-follower MAS.
Proof: q x is upper bounded by a finite value in the following way.
where ε 0,max = max t e x,0 and ε d,max = max t d x . Since the finiteness of ε 0,max and ε d,max follows from the assumption-4, q x has the finite value which means the asymptotic bounded consensus.
One interesting result can be found when every communication delay does not change with time as follows.
Proposition 3: The consensus control given in (20) for a MAS does not require delay information when every communication delay is fixed as a constant.
Proof: When every communication does not change with time,τ ij (t) =τ ij (t) = 0 for all i and j. Thus, (20) is It is noted that when every communication delay does not change with time, the sliding mode consensus control can achieve the asymptotic bounded consensus with sharingẍ j only.

B. A SLIDING MODE CONSENSUS CONTROL WITH UNKNOWN DELAYS
Communication delay information is often unavailable or inaccurate. In this case, the consensus control presented in (20) cannot be applicable since it requires information on the communication delays. Thus, we propose a consensus control for a MAS with unknown delays. Since the system model does not change with the availability of the information on the communication delay, the same topology dependent disagreement vector in (18) and the same sliding variable in (19) will be used to develop a consensus control. The consensus control for a MAS with unknown time varying delays and disturbance is presented in the following theorem.
Theorem 4.2: Let the proposed consensus control u i be given by Then, s i converges to 0 in a finite time.
Proof: Let a positive definite Lyapunov function V i be given by 0.5s 2 i .ṡ i in (21) can be rearranged aṡ (27) where u i = u e,i + u s,i . v x,i can be considered as an additional disturbance following from the uncertainty in delay information. By setting u e,i as −cẋ i + ca −1 where k u > max i,t |v i | + max i,t a −1 i,sum |v x,i |. Inserting (28) and u e,i into (27) verifies thatV i < 0 with the similar arguments made in the proof of the theorem-3.1. The proof of the convergence in a finite time can be also proved by following the same arguments made in theorem-3.1.
It is noted that (26) does not require information on delay. However, it may have to pay more control effort at the expense of being unequipped with information on delays. Since it does not use delay information, the required information flow from neighbor agents will be less than with known delay. In addition, the consensus control requires the knowledge of the first-order dynamics from neighbor agents only. The properties of the proposed consensus can be summarized by the following proposition.
Proposition 4: The consensus control given in (26) for a leader-follower MAS with unknown time varying communication delays achieves the asymptotic bounded consensus. Proof: The consensus control given in (26) results in the same dynamics in the sliding surface as with known delay. Thus, q x is upper bounded by a finite value, which proves the proposition.

V. SIMULATION RESULTS
To validate the theoretical results in the previous sections and assess the characteristics of the proposed consensus control, a simple hypothetical MAS with 1 leader and 4 followers is considered. The simulation configuration can be considered as the instance of coordinated movement of vehicles moving along a single axis [34] or mobile robot coordination [35]. The corresponding communication topology is given in figure-1 where the index of the leader agent is 0. Unless otherwise stated, the leader dynamics followsẍ(t) = cos(3t) without disturbance while disturbances for followers are given asẍ i (t) = sin(i · t). c is set as 1. The initial state of each agent is given as , anḋ x(0) = 0. It is assumed that the consensus protocol starts at t = 1 to deal with delayed information. However, when the trajectories are plotted with the time being shifted to −1 so that the start time of consensus protocol can be 0.
We first consider a time varying delay with non-zero higher-order derivatives. τ ij (t) is set as 0.25 · cos(t + ϕ ij ) + 0.35 where ϕ ij is generated from a uniform distribution over [0, 2π ]. Both k u and k u were set as 2. Thus, delay dynamics are identical while their values are different at the same instance. The corresponding delays change periodically from 0.1 to 0.6. The figure-2 shows the state trajectories resulting from (20) and (26). It shows that both consensus protocols achieve asymptotic bounded consensus. It is noted that the resulting trajectories are very similar while the consensus protocol with unknown delays uses limited information. (20) can be rearranged as in the following form.
The first three terms are the same as the consensus control with unknown delays. With setting k u and k u as the same value,ṡ i becomes the same for both controls, It implies that both sliding variables may have the same dynamics, which can be confirmed from the figure-3. However, the state trajectories can be different since it has different control inputs. Even though state trajectories appear to be identical, they are slightly different. The figure-3 also verifies the finite time convergence of the sliding variable which was stated in the theorem-4.1 and the theorem-4.2.  The same numerical simulation was done with a different type of delays in figure-4. τ ij (t) is set as t for every communication path. Both k u and k u were set as 2. The figure-4 also verifies the asymptotic bounded consensus. For this specific case, all agents receive the same information which is information at the initial time. Consequently, every follower agent is found to converge to a constant value. It is also observed that agent 3 and agent 4 converge to the same value as they receive the same information only from agent 2. Since the convergence of the sliding variables was similar to one in figure-3, the corresponding plot was omitted.
The consensus performance is likely to depend on the maximum delay. To assess the effect of the maximum delay, the performance of the proposed consensus control was evaluated with the average consensus error which is defined where T is the terminal time of the simulation which is 20 secs. τ ij (t) is set as τ ij (t) = γ (σ (t) − 0.5) for every communication path where σ (t) is a sigmoid function, and γ is a parameter determining the maximum delay. Both k u and k u were set as 2. The resulting VOLUME 9, 2021  average consensus errors for different delays are found to be almost the same for the consensus controls with known delay and unknown delay. This can be expected from the results presented in the figure-2 and the figure-4. The upper bound of the disagreement vector was shown to increase with the change in states over delay time in (25). From this result, the average consensus error is expected to increase with the maximum delay, which is shown in the figure-5. γ increases from 0.05 to 2 by 0.05 step in the figure-5. When the maximum delay is less than 1, the average consensus error is observed to increase with the maximum delay up to some point and decrease with the maximum delay. This result is conjectured to be due to the periodicity of the excitation signal in the leader agent.
The upper bound of the disagreement vector in (25) depends on the maximum change of the state of the leader agent for a fixed delay. To assess the effect of the magnitude change in the leader state, the leader dynamics is given asẍ(t) = β cos(3t) where β is a parameter determining the degree of the change for a given time interval. τ ij (t) is set as 0.25 · cos(t + ϕ ij ) + 0. 35. The figure-6 shows the average consensus error for increasing β from 0.5 to 10 by  (24) is proportional to the amount of change in the state of the leader agent for a given delay. From this observation, it is expected that the average consensus error is likely to be proportional to β, which is clearly shown in the figure-6. While the consensus control for the known delay with k u = 2 achieves the convergence of sliding variable regardless of β, the consensus control for the unknown delay with k u = 2 does not converge for large β which results in a larger average consensus error. When k u = 15 it achieves almost the same average consensus error as the consensus control with the known delay. The consensus control for the known delay is shown to make no difference as long as k u satisfies the convergence condition. It is observed that the consensus control for the unknown delay with k u = 2 provides a lower average consensus error than the consensus control for the known delay when β < 5. When e i is small, the average consensus error is usually small. e i = 0 does not guarantee the smallest average consensus error when there is a communication delay. It is conjectured that the consensus control for the unknown delay with k u = 2 benefits from error incurring from incomplete convergence of sliding variables. However, when e i is larger than some value, the average consensus error will be proportional to the magnitude of e i , which explains that larger average consensus error when β > 5.
The proposed consensus control was compared with the existing state of art method [26] in figure-7. Due to the limited applicability of the control in [26] which was developed for a MAS with a time-varying state delay, every communication delay on the link in the figure-1 was set as 0.5 cos(t)+0.5. The velocity of the leader agent was set as the constant velocity of 20m/s, since the consensus control in [26] was developed with the assumption that the leader moves at a constant velocity. When disturbance is not large, the proposed control shows comparable performance to the control in [26]. It is noted that the consensus control [26] assumes a priori known information on the velocity of the leader agent at every follower agent. The proposed control shows the robustness to the increased disturbance while the consensus of the comparing control degrades significantly.

VI. CONCLUSION
In this paper, a sliding mode consensus control for a MAS with unknown time varying delays has been proposed. The proposed method exploited the properties of the sliding mode control through considering delay-related terms as a disturbance. It was shown to achieve the asymptotic bounded consensus and finite time convergence of a sliding variable through sharing the derivative of the state with neighbor agents. The numerical verification showed that the proposed method almost achieved the same level of consensus as the consensus control with known delays even when a time varying delay was unknown and very large.
There are several future research directions which need to be paid attention to. The proposed consensus control requires the measurements of the first order and the second order derivative of states. However, measurement noise due to quantization or thermal noise can degrade performance. Thus, developing an algorithm robust to the measurement error will be necessary to make the proposed algorithm to be applicable to a practical system. Even though the proposed algorithm guarantees the asymptotic bounded consensus when a parameter is set to satisfy a condition, the explicit condition needs to be made further. A conservative condition will be easily found when the dynamics of each agent vary within some bounds. The predictive control can be also exploited to reduce the consensus error. Practical consideration needs to be made further to be applicable to a real physical system. The control signal is usually generated in a digital domain. Thus, the control signal is likely to be piece-wise continuous. In addition, state information and observation will be also discretized. When agents are heterogeneous, their processing speed can be different, which may result in different sampling periods for each agent. This heterogeneity can be further considered when the agents are connected through different networks, which have different unit transmission period. Moreover, the notorious chattering problem with sliding mode control needs to be also studied with some exiting methods such as super-twisting sliding mode (STSM) control [36], and a composite STSM control with disturbance observer to reduce gain [37], [38]. To deal with uncertainties, alternative methods based on artificial intelligence such as reinforcement learning [39], Fuzzy control [40], and Neural network [41] can be considered to improve the robustness of consensus control.