Distributed Consensus Control of Networked Robotic Systems With Dynamic Leader Under Time-Varying Delays and Denial-of-Service Attacks

Consensus of multi-agent systems has a wide range of applications, such as coordination, cooperative transportation, and synchronization. This study investigates consensus control in the task space of multi-robotic systems modeled using Euler-Lagrange equations with a moving leading under time-varying delays and denial-of-service (DoS) attacks, respectively. Two control algorithms are studied when networked robotic systems are interconnected over directed spanning trees with a constant-velocity leader as the root. First, we use the Lyapunov-Razumikhin theorem to address a condition of the maximum time delay for nonuniform time-delay communication, thereby ensuring asymptotic tracking of the network. Second, we propose a novel resilient control framework to deal with DoS attacks. Note that designing the control algorithm for DoS attacks is difficult because the attacks cause signal discontinuity. The proposed resilient controller can release the DoS duration condition, which is a common requirement in many related works. It is verified that the consensus errors converge to the origin under the proposed control frameworks. We present experimental results to demonstrate the effectiveness of the proposed controllers.


I. INTRODUCTION
In recent years, numerous control algorithms have been developed for multi-agent systems resulting from their potential use in cyber-physical systems (CPSs), such as autonomous vehicles, networked robotic systems, and power grids. A plethora of control algorithms have been proposed to address multi-agent system problems, such as consensus under a dynamic leader with time delays or time sampling [1], [2], [3], [4], [5], [6], and security controls against malicious/adversarial attacks [7], [8], [9], [10], to name a few. However, most of the proposed controllers are for linear systems and might not be directly applied for nonlinear systems, especially those modeled by using the Euler-Lagrange formulation. Noted that a wide class of mechatronic systems The associate editor coordinating the review of this manuscript and approving it for publication was Engang Tian .
can be described by the Euler-Lagrange equation, such as mobile robots, manipulators, mobile manipulators, and quadrotors. The limitations of these aforementioned studies and the importance of Euler-Lagrange systems, motivated us to expand previous results, mostly from [2] and [9], to apply to networks of multiple Euler-Lagrange systems.
In the literature, the synchronization of networked Euler-Lagrange systems has been studied under time delays with switching topologies [11], jointly connected topologies [12], uncertain kinematics/dynamics of robotic systems [13], and model-free optimal consensus control [14]. However, the aforementioned control methods require the leader information to be available to all followers, which may not always be feasible in reality owing to dynamic connection or resource limitations of networked systems. Several control algorithms have been presented in [15], [16], [17], and [18], where only a subgroup of followers can obtain the leader VOLUME 10, 2022 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ information and the others use local information to achieve consensus. Namely, [15] addressed the leader-follower consensus problem for networks of Lagrangian systems without using task-space velocity, and the problem is studied under time-varying samples [16], time-varying delays [17], and event-based samples [18]. However, the leaders in [15], [16], [17], and [18], are immobile, thereby indicating that the consensus are achieved from one point to another point. Therefore, they may be inefficient in many applications of networked robotic systems. Until recently, consensus control algorithms with a dynamic leader have been studied under delay-free communication [19], [20], discrete-time communication [21], and the identical constant time delay [22].
To the best of our knowledge, consensus control of networked Euler-Lagrange systems with a dynamic leader, which is only available to a subgroup of followers, under nonuniform time-varying delays has not been studied in depth.
In control of networked systems, besides communication delays, cyberattacks is also another problem that needs to be considered [23], [24], [25]. Among cyberattacks, denialof-service (DoS) attacks is common, effective, and simple offense techniques in CPSs, where an attacker attempt to prevent networked transmission so that the target agents or networks cannot provide normal services [26]. As a result, considerable secure control laws against DoS attacks have been developed. For instance, the conditions for guaranteeing the input-to-state stability of networks under DoS attacks have been studied in [27] for linear systems, in [28] for nonlinear systems, and in [29] for multi-agent systems. Additionally, in [30] the authors utilized switching topologies to avoid or handle DoS attacks. Currently, research on Euler-Lagrange systems with DoS attacks is limited [31], [32]. The design of resilient controllers for networks of multiple uncertain Euler-Lagrange systems, that guarantee tracking under DoS attacks remains a difficult problem because of the discontinuity of signals due to the attacks.
Motivated by the aforementioned observations, in this paper, we propose two control frameworks for networked Euler-Lagrange systems to achieve tracking with a constant-velocity leader under nonuniform time-varying delays and DoS attacks, respectively. Hence, there are two challenges for solving these problems. 1 The results of existing works cannot be directly implemented for networks of Euler-Lagrange systems with of moving leaders and timedelay communication. 2 Due to DoS attacks, the communications between robots are discontinuous, thus, the control algorithm must be carefully developed to avoid unwanted impulse signals. The main contributions of this study are listed as follows: • Different from [19], [20], and [21], the proposed control algorithm in this work guarantee asymptotic tracking under nonuniform time-delay communication, where coupling delays are unavoidable in multi-agent systems.
In contrast to the works in [2], [4], [5], and [6], where the system dynamics is linear, the system dynamics in this study is modeled by the Euler-Lagrange equation, which is highly nonlinear.
• A novel resilient control algorithm is proposed to handle discontinuous signals due to the DOS attacks. In addition, the proposed control algorithm ensures asymptotic stability under the attacks. Moreover, in contrast to most of the works on DoS attacks, which assume the attacks have limited energy so that the frequency and duration of the attacks are bounded, our proposed controller only requires that the DoS frequency satisfies the average dwell-time condition [33].
• The proposed control algorithms are distributed to maintain the autonomous nature of networked multi-agent systems, thereby allowing the expansion of the network without extra complexity. Additionally, experimental demonstrations are provided to validate and demonstrate the efficacy/potency of these proposed control algorithms. The rest of this paper is organized as follows. Research background and problem formulation are addressed in Section II. The main results are presented in Sections III and IV. Illustrative examples are addressed in Section V. Finally, Section VI concludes this paper.
Notation: · and · F are used to denote the Euclidean and Frobenius norms. | · | denotes the cardinality. I n i denotes n i ×n i identify matrix. diag{a 1 , a 2 , . . . , a n } represents a diagonal matrix of n dimension. λ min (P) and λ max (P) present minimum and maximum eigenvalues of the enclosed matrix P. ⊗ is the Kronecker product.

A. MODELING OF ROBOTIC SYSTEMS
Consider a leader-follower networked robotic system, which consists of a virtual leader and N followers. The leader information (i.e., X 0 (t) andẊ 0 (t) ∈ R p ) are provided in the work space, whereẊ 0 (t) is a constant. The i th follower is modeled by Euler-Lagrange formulation such that where q i ∈ R n i is the vector of generalized space, M i (q i ) ∈ R n i ×n i stands for the inertia matrix, which is positive-definite, C i (q i ,q i ) ∈ R n i ×n i presents the Coriolis/Centrifugal force matrix, G i (q i ) ∈ R n i denotes the gravity vector, and τ i ∈ R n i represents the applied torque vector. From Euler-Lagrange structure [34], there are several useful properties for (1). Property 2 (Skew Symmetry): With the appropriated con- Let X i (t) = f i (q i ) : R n i → R p represent a mapping from the joint space to the task space of the i th robot. Additionally, we haveẊ representing the mapping of the velocity from the joint space to the task space. It is noted that the dimension p of the task pace is identical, but the joint-space dimension n i could be nonidentical for each robot. J i (q i ) ∈ R p×n i denotes the Jacobian matrix. Considering that the inversion of J i (q i ) is bounded and existed.

B. GRAPH THEORY
The interconnections in networked systems are described by a directed graph G = {V, E, A}, where V = {η 0 , η 1 , η 2 , . . . η N } denotes the set of finite N + 1 nodes (e.g., node η 0 represents the leader, and node η i represents the then G is called undirected graph, otherwise called directed graph. A graph contains a directed spanning tree if there exists at least a node (root) that connects all other nodes by directed paths. A directed path from node η j to node η i is a sequence of edges of the form If G contains a directed spanning tree rooted at η 0 , then the Laplacian matrix L can be formulated as 10 , a 20 , . . . , a N 0 }, and L F denotes the Laplacian matrix of G F . The Laplacian matrix of a directed spanning tree graph has only one simple zero eigenvalue, and the others are strictly positive [2]. Thus, L meets a property such that all eigenvalues of L 22 have positive real parts [19]. Assumption 1: The graph G has a directed spanning tree rooted at node 0.

C. PROBLEM FORMULATION
Given a virtual leader X 0 (t) in the task space, whereẊ 0 (t) is a constant, and the leader information is available to a subset of the followers (1). We are going to develop two distributed control schemes for the followers to guarantee asymptotic tracking of the leader, such that under time-varying communication delays and DoS attacks, respectively. The following lemmas are provided to support analysis.

is a Lipschitz continuous function with respect to its argument. If there exists a continuous function V (x, t) which satisfies
and exists a continuous, positive, and nondecreasing function then (3) is uniformly asymptotically stable at the origin. Lemma 2 [18]: Consider a linear time-varying systeṁ where x ∈ R n is the system state, A(t) ∈ R n×n represents a uniform bounded matrix, and s, d ∈ R n represent external input vectors. Suppose thatẋ = A(t)x is uniformly exponentially stable.

III. CONSENSUS UNDER TIME-VARYING DELAYS
The control algorithm for networked Euler-Lagrange systems achieving consensus with a moving leader under asymmetric communication delays is addressed in this section. Let us start by designing an estimator to obtain the velocity of the leader such that [2], [36] where K is a positive constant, and T ij := T ij (t) is a bounded time-varying communication delay of the signals transmitted from the j th agent to the i th agent. Considering that 0 < T ij (t) ≤ T max < ∞ andṪ ij (t) < 1, which implies the signals are received in order [37], [38].
Proposition 1: Consider a leader having constant velocity and existing directed paths from the leader to all followers. By using (7),v i converges toẊ 0 exponentially.
The proof of Proposition 1 is presented in Appendix. Next, based on the estimation result, we propose a control algorithm for the networked Euler-Lagrange systems achieving tracking in the task space. Let us define the reference velocity in the task space for the i th robot such that where the time delays T ij and T i0 can be obtained from the time stamped data. From (8) then we can get the reference signals in the generalized space and auxiliary variables aṡ where By utilizing the linearity property, the consensus control law is constructed as where· denotes the estimation of robot parameters, K x i , K q i , and i are positive-definite diagonal matrices. Substituting (14) into (1), we get the closed-loop dynamic where˜ i =ˆ i − i represents the deviation in estimating the parameters of the robot. The following theorem summarizes the control algorithm of this section. Theorem 1: Consider a networked multi-agent system described by (1), the graph G contains a directed spanning tree rooted at node η 0 which has a constant velocity, and the communication delays have the upper bound T max such that Under the proposed control framework, which is designed in (7) and from (8) to (15), the networked robotic system achieves consensus in the task space asymptotically. Proof: Consider a Lyapunov-like function such that According to Property 2, update law (15), and s X i = J i s i , the above equation yields tȯ which implies that s i , s X i ∈ L 2 ∩ L ∞ , and˜ i ∈ L ∞ . Moreover, Propriety 1 and (16) verify the boundedness ofṡ i . According to Barbalat's lemma, s i and s X i = J i s i asymptotically converge to the origin. In the following,Ẋ i and X i converging toẊ 0 and X 0 asymptotically is demonstrated. By substituting (8) into (13), we obtain T w denotes a set of time delays with w = 1, 2, . . . , m, m = |T w | represents a cardinality of set T w , and A w represents a set of adjacency matrices corresponding set T w , such that all elements of A ij are set as zero except for a xy = a ij . Hence, we have representing the collective dynamic of (20) Next we are going to show that the systeṁ is exponentially stable at the origin if the condition (17) satisfy. For the compactness, we denote Q = K (Q ⊗ I p ) and A w = K (A w ⊗ I p ). From the formula of Newton-Leibniz yields that (26) can be rewritten aṡ As the leader has directed paths to the followers, L 22 is Hurwitz matrix. Thus, there exists a positive-definite matrix P ∈ R N ×N such that L 22 P + PL T 22 = I N . Let us consider a Lyapunov Razumikhin candidate function as V R =X T (P ⊗ I p )X . It is obvious that λ min (P) X 2 ≤ V R ≤ λ max (P) X 2 . By differentiating V R along the trajectory of (28), we obtain thaṫ From the inequality 2x T y ≤ x T x + y T −1 y, where x, y ∈ R n and ∈ R n×n is a positive-definite matrix, we get thaṫ It is noted that T w ≤ T max which implies that if T max satisfies (17) thenV R ≤ −δX TX for some δ > 0. It indicates that X ∈ L 2 ∩ L ∞ . Moreover, by invoking Lemma 1, we verify that the system (28) is asymptotically stable at the origin. It is noted that (28) is equivalent to (26), a linear systems. The asymptotic stability of linear systems are equivalent to the uniform exponential stability. Therefore, we conclude that system (26) is exponential stable for ∀T w ≤ T max . Consequently, the system (25) is input-to-state stable with external input s X + v. Additionally, we have shown that both s X andv belong to L 2 ∩ L ∞ and converge to the origin. Thus, X andX converge to the origin when s X andv converge to the origin according to Lemma 4.8 in [39].

IV. CONSENSUS UNDER DoS ATTACKS
In this section, we develop a resilient control framework for networked robotic systems against adversarial inputs in cyber-physical systems that are denial-of-service (DoS) attacks on the communication channels. We assume that there exist no delay communications in this section. Consider the distributed DoS attacks, where the attacks on each channel are independent. When DoS attacks occur, the transmission channels are blocked causing agents to be unable to update their neighbors' information. It is assumed that the DoS attacks can occur at any time with any duration but have finite frequencies.
Remark 1: Different than [8], [40], [41], where the DoS attacks are considered to block all the communication channels when active, this work considers that an attacker can disrupt all or several communication channels of the network at a time. Thus, (t) contains 2 |E| 2 different attack modes (e.g., several attack modes are depicted in Fig. 1). Furthermore, this work proposes a control algorithm that can handle DoS attacks, while [8], [40], and [41] studied conditions of the attacks that not affect to the system stability.
The observation of the leader velocity in the case DoS attacks is constructed such thaṫ whereâ ij = a ij if (j, i) / ∈ (t) without attack,â ij = 0 if (j, i) ∈ (t) under attack, andv s 0 =Ẋ 0 . Remark 2: It is noted that the attack modes, (t), are random and unknown to the robots in the network. However, (33) is feasible because if j ∈ N i then whether (j, i) ∈ (t) or not can be identified by robot i. For instance, if the i th agent receivesv s j then (j, i) / ∈ (t), otherwise (j, i) ∈ (t). VOLUME 10, 2022

Proposition 2: Consider a leader moving with a constant velocity, if there exists a positive diagonal matrix P such that
then by using the observation (33),v s i will converge tȯ X 0 exponentially.
The proof and analysis of Proposition 2 is studied in Appendix. Noted that the discontinuous signals which occur when the attacks changing between on/off is the main challenging in designing resilient control for networks of uncertain robotic systems. The control algorithm, which was designed from (8) to (15), can not be used here due to DoS attacks. Namely, if we applied (8) and (9) in this case, we would have v r i =v s However,ȧ ij andȧ i0 are not finite at the attacked points, which imply the unbounded ofv r i . Note that resilient controls to DoS attacks for consensus of networked Lagrangian systems have been studied in [31] and [32]. Using results of [31] and [32],v r i can be designed such thaṫ whereφ i = −ϕ i + 1 N i j∈N iâ ij X j , K 1 , K 2 are positive control gains, andN i = j∈N iâ ij stands for cardinality of N i .v r i designed in (36) can handle the problem of discontinuous signals. However, it is only useful in the case of the immobile leader. (36) cannot be directly applied for the case of the dynamic leader. Beside, both DoS duration and frequency need to be constrain in [31] and [32]. In this paper, the condition of the DoS duration is released by utilizing the observation. Thus, we propose a new control framework to overcome the problem. For compactness, let us define an auxiliary variable as where is a positive scale value. The design of reference acceleration in the task-space which against DoS attacks is proposed such thaṫ v s Hence, we design the reference signals in the joint space and auxiliary variables aṡ The control input is computed such as The control framework, which is resilient against DoS attacks, is summarized as in Theorem 2. Theorem 2: Consider a network of multiple robotic systems described by (1), the graph G contains a directed spanning tree, and the root has a constant velocity. Under the proposed control framework which is given in (33) and from (37) to (44), the network will asymptotically achieve task-space consensus in the presence of DoS attacks if condition (34) is satisfied.
Proof: In the proof of Theorem 1, we have shown that by choosing the Lyapunov-like candidate function Thus, according to Lemma 2 in [18], we have lim t→∞Vi = 0, which indicates that s X i asymptotically converging to the origin. Now we verify that the network robotic system achieves task-space consensus in the presence of DoS attacks.
By definingū i = u i − u 0 ands X i = s X i − s X 0 with i = 1, 2, . . . , N . Thus, for the network of followers we obtain thaṫ . . ,ṡ T X N ] T . From the proof of Proposition 2, we conclude that the linear time-varying systeṁ is uniformly exponentially stable. In addition, Consequently, according Lemma 2, we verify thatū ∈ L 2 .
The definition ofū i = u i − u 0 and (37) yield which is the exponential stable linear system with external inputū i ∈ L 2 . According to Lemma 4.8 in [39], we have (Ẋ i −Ẋ 0 ) ∈ L 2 , (X i −X 0 ) ∈ L 2 ∩L ∞ , and lim t→∞ (X i −X 0 ) = 0. In addition, (38) can be rewritten such thaṫ v s Obviously, K j∈N iâ ij (u i − u j ) − s X i = K j∈N iâ ij (ū i − u j )− s X i ∈ L 2 andv s r i = − v s r i is the exponential stable linear system. Hence, we get lim t→∞ v s r i = 0. Furthermore, (42) can be rewritten asẊ we can conclude that lim t→∞ (Ẋ i −v s i ) = 0 according to lim t→∞ v s r i = 0 and lim t→∞ s X i = 0. By invoking Proposition 2, we get that lim t→∞ (Ẋ i −Ẋ 0 ) = 0.
In summary, the aforementioned analyses have verified the system stability and shown lim t→∞ (X i − X 0 ) = 0 and lim t→∞ (Ẋ i −Ẋ 0 ) = 0. Hence, the robotic network achieves task-space consensus.
In the proposed control framework, (33) and (43) are discontinuous due to DoS attacks. Therefore, the solutions should be understood in terms of the Filippov solution [42]. However, (33) and (43) are measurable and bounded, in addition, in the proofs, the Lyapunov functions are continuously differentiable. Hence, the proofs are verified similar the case of continuous systems.
Remark 3: The discontinuous signals occurred when the DoS attacks changing between on/off is the main challenging in designing resilient control for uncertain Euler-Lagrange systems. Under the proposed control framework, we can avoid impulsive behaviors which are caused by the attacks and can achieve consensus with a dynamic leader.
Intuitively, (34) is satisfied if the network has a directed spanning tree topology rooted at the leader, for example, the connections in Fig. 1 (c) and (d) still contain directed spanning tree even under the attacks. In case of after DoS attack, the network does not have spanning tree (e.g., the attacks modeled in Fig. 1 (a) and (b)) a predefined connectivity restoration mechanism or switching mechanism should be activated to create new links for restoring the directed spanning tree condition. Note that the connectivity restoration mechanism can be independently activated on each agent by checking neighbor's conditions. For instance, in the case of Fig. 1 (a), all agents will take action to create new connections because they all have no neighbors. In the case of Fig. 1 (b), both agents, 3 and 4, need to request the restoration mechanism. Because agent 3 has no neighbors, and agent 4 has neighbors but can not receive signals of the leader (in this case, the leader and the agent which connects to the leader are predefined). In the case of Fig. 1 (c) and (d), there is no need for requesting the restoration mechanism because agent 4 still has the leader as its neighbor, and agent 2 also has a neighbor. In the case of applying switching communication to handle the DoS attacks, the following assumption is considered to guarantee the stability of switched systems.
Assumption 2: The DoS frequency is satisfied the average dwell-time τ * D , which was addressed in [33]. Namely, let (t 1 , t 2 ) represents the total number of the on/off transitions of DoS attacks occurring in a time interval [t 1 , t 2 ). For any average dwell-time τ D ≥ τ * D > 0, the condition (50) is satisfied for all 0 ≤ t 1 < t 2 and given positive numbers Remark 4: The proposed control framework can be directly applied for systems under switching communication topology with trivial modification such thatâ ij in (33)   (34) is not a tricky one as it always satisfies under directed spanning tree graphs. In addition, the proposed control method can be directly applied for switching communication to guarantee the spanning tree topologies of the network robots.

V. EXPERIMENTAL EXAMPLES
The experiments using a virtual leader and four Touch Haptic devices are presented in this section. The connection topology and the robots are depicted as shown in Fig. 2. The control programs are written on a desktop computer by using C ++ with 1kHz of sampling rate.
The virtual leader moves on the X-Y plane with a constant velocity, 1 mm/second, such as X 0 (t) = [−40; −110; 0] + [1t; 1t; 0t] mm. The time delays are selected as T 14 = 0.2 + 0.05sin(t) second, T 21 = 0.2 + 0.05cos(t) second, T 32 = 0.15 + 0.01cos(t) second, T 43 = 0.15 + 0.05sin(0.5t) second, and T 24 = T 42 = 0.2 + 0.05(sin(t) + cos(t)) second. The control parameters are chosen as K = 5,  (3,4) without using the proposed resilient control algorithm. The attacks start at time 30th second and continuously block the channel to the end of the experiment. It is clearly observed that there is a big divergence between the followers and the leader. Fig. 6 demonstrates the results under the proposed resilient control framework for the case an attacker blocks channels (1, 2) and (3, 4) at time 30th second.    The results show that all the followers smoothly achieve consensus with the leader, which indicates the practical usefulness of the proposed control algorithm. Remark 6: The proposed resilient control algorithm guarantees bounded signals even if the attacker blocks all channels. In this case, (46) is equivalent tou =ṡ X that implies u i =s X i +ū i (t a ) ∈ L ∞ , where t a is a time when the attack occurs. Thus, from (48), we have Ẋ i −Ẋ 0 ≤ s X i + ū i (t a ) and X i −X 0 ≤ ( s X i +ū i (t a ) )/ that indicate the boundedness of tracking errors. Furthermore, from (49), we can also show thatẊ i converge tov s i (t a ). The experimental results which is shown in Fig. 7 is for the case an attacker blocking all the channels at 30 second to the end of the experiment. The results indicate that the proposed resilient control ensures tracking quality even if the attacker blocks all the communication channels for a long time.

VI. CONCLUSION
This paper explores two control algorithms for networks of multiple Euler-Lagrange systems tracking a virtual dynamic leader under nonuniform time-varying delays and denial-ofservice (DoS) attacks, respectively. Given that the leader has constant velocity and directed paths to followers, a condition for achieving consensus under nonuniform time-varying delays was established on the basis of the Lyapunov-Razumikhin technique. Furthermore, a novel resilient control framework was proposed to handle DoS attacks and discontinuous signals due to the attacks. It is verified both in theory and experiment that the proposed control algorithms guarantee the asymptotic stability of the network. Future work will encompasses the development of the resilient control algorithm with time-varying delays, and analyses computational complexity of the algorithms.

APPENDIX
Proof and Analysis of Proposition 1: As the leader has constant velocity we get thatẊ 0 (t − T i0 ) =Ẋ 0 (t) = constant. By definingv i =v i −Ẋ 0 , (7) can be reformulated aṡ The authors in [36] have proved that the system (51) is uniformly and asymptotically stable at the origin. Since asymptotic stability of linear systems is equivalent to the uniform exponential stability, we conclude thatv → 0 exponentially, which indicatesv i →Ẋ 0 exponentially. Condition (34) guaranteesV D ≤ 0, so (52) is asymptotically stable andv s ∈ L 2 ∩ L ∞ , which impliesv s ∈ L 2 ∩ L ∞ . Since asymptotic stability of linear systems is equivalent to uniform exponential stability,v s goes to the origin exponentially [38], namely,v s i converges toẊ 0 exponentially.  33 He is currently a Professor with the Depart-34 ment of Mechanical Engineering, National 35 Cheng Kung University (NCKU), Tainan, Taiwan. 36 His research interests include control of networked robotic systems, bilateral 37 teleoperation, multi-robot systems, mobile robot networks, and human-robot 38 interaction.