Couple-Group Consensus of Heterogeneous Multi-Agents Systems With Markov Switching and Cooperative-Competitive Interaction

In this paper, couple-group consensus is investigated for a kind of heterogeneous multi-agent systems (HMASs) with Markov switching. Some novel couple-group consensuses have been proposed, in which cooperative-competitive interaction, Markov switching and time delay are all considered. For Markov switching, the transitive rate of probability can be divided into two cases: known or partly known. Based on stochastic delta operator, probability, graph and stability theories, the leader-following and pinning couple-group consensus of this system be converted into analyzing the stability of related switching delta operator system. In the obtained results, some conservative conditions, such as the balance of in or out degree, strong connectivity and containing a spanning tree, can be longer strictly demanded. Some numerical examples are given to show the validity of the acquired results.


I. INTRODUCTION
Recent years, consensus has become a hot issue in distributed control of MASs due its promising applications in many domains, such as the formation of mobile vehicles [1], [2], distributed control in sensor networks [3], [4], group decision-making systems [5] and so forth. The mainly ideal of consensus is to design an effective protocol to make all agents reach a same state. As an extension of consensus, group (cluster) consensus means that agents in the whole system can be divided into several groups. Agents in different groups will reach different states. Agents in the same group will reach same state. In some extent, consensus or synchronization, is a special case of group consensus. In fact, The associate editor coordinating the review of this manuscript and approving it for publication was Dominik Strzalka . group consensus is very common in real life [5], [6], [7], [8], [9], [11], [12]. Hence, it is important for us to investigate the group consensus of MASs. So far, a lot of meaningful achievements have been acquired for the group consensus of MASs [6], [7], [8], [9], [10], [11], [12]. In [6], pinning control is used to realize group tracking consensus of 2nd-order nonlinear MASs. In [7], based on cooperative-competitive relation, some sufficient conditions have been obtained for the group consensus of a kind of HMASs. In [8] and [9], group consensus of MASs with switching topology and time delay has been discussed. In [10], distributed PID controller is used to discuss the consensus of first-order MASs with time delay. In [11] and [12], group consensus of 2nd-order or even high-order MASs have been studied respectively. It worth pointing out that most of obtained results are needed to suppose the topologies of MASs are homogeneous. In a homogeneous MASs, all agents have the same dynamics. However, in many practical cases, it is impossible for each agent to have the identical dynamics. Hence, it is essential for us to investigate the consensus of heterogeneous MASs [13], [14], [15], [16], [17]. In [13], Sun and Zhang have focused on the output sign consensus problem of HMASs with fixed or switching signed graphs. In [14], time-varying output formation tracking problem of heterogeneous linear MASs has been concerned. More correlative results can be found in [15], [16], and [17]. On the other hand, from the view of control cost, it is quite difficult for us to control all nodes to achieve consensus in many large-scale complex networks. For this purpose, the pinning control method has been proposed. According to existing results, the pinning control can be used to realize the consensus or synchronization of MASs [18], [19]. In [18], event-triggered strategy and pinning control have been used to achieve consensus of MASs. In [19], predictive pinning control has been applied to obtain the consensus of MASs with time delay. In [20], pinning control is applied for the leader-following synchronization of MASs. Different from other control protocols, it is not necessary for us to control all nodes in pinning control strategy. The obtained results have also shown that the pinning control scheme can reduce the control cost. Furthermore, most of the topology structures of MASs are supposed to be fixed when we discuss consensus of these systems [21], [22]. Nevertheless, in most practical cases, the topology of MASs may change with time varying, it is not fixed [23], [24], [25]. Thus, it is also more significant for us to investigate the consensus of MASs switching topology [23], [24], [25], [26]. However, among most existing results, the switching ways are usually assumed to obey some specific rules such as ADT or MDADT [24], [26]. In [24], MDADT method has been used to address the consensus problem of HMASs with switching topology. In [26], based on ADT, switching scheme has been proposed for compensating packet dropout. In fact, various internal and external disturbances may exist in real system, such as uncertain distance, time delay, connection failure and so on. All these reasons can lead to different switching ways. Sometimes, these switching ways are even stochastic. In this situation, it is very difficult for us to use the novel mathematical models or theories to analyze the performance of these systems. How to describe these various switching manners? How to analyze the couple-group of these systems with switching, especially stochastic switching manner is considered? It is very necessary for us to establish new mathematic model and introduce novel analytic tool. Fortunately, Markov process has excellent merit in describing uncertain abrupt phenomena. It has been widely used in robust detection filters, image recognition, and other fields [27], [28], [29]. Therefore, it is natural for us to introduce the Markov process to describe MASs with stochastic switching manner. Of course, the transitive probability matrix of Markov process may not be fully known in many situations. It is very significant for us to analyze the consensus of MASs under Markov switching with part known transitive probability. In this paper, two cases of Markov process with known or unknown transitive rate matrix will be discussed when we investigate the couple-group of HMASs.
Moreover, most of the acquired achievements of MASs must obey the assumption that relations among agents are cooperative or competitive [30], [31], [32], [33]. However, cooperative and competitive relation often coexist in natural world. Therefore, it is essential for us to study the related problems of MASs with cooperative and competitive relations [7], [24]. As we know, delta operator is an effective tool in analyzing the performance of a system whatever it is discrete or continuous [34], [35], [36], [37], [38], [39]. In [37], the delta operator has been firstly proposed to unify the discrete and continuous system into a single framework. In [38], the delta operator has been used to detect observerbased fault. In [39], the delta operator has been applied to analyze sliding mode control. Undoubtedly, based on delta operator, more and more meaningful results have been derived [27], [40]. According to existing literature, the delta operator can also be applied to study the consensus problem of MASs [24], [41]. In [24], the delta operator has been used to handle the group consensus problem of HMASs in cooperative-competitive networks. And in [41], the delta operator has been utilized to discuss the consensus problem of MASs with faults and mismatches. Inspired by the above discussions, this paper will use pinning scheme and stochastic delta operator to investigate the couple group consensus of HMASs with cooperative-competitive relation and Markov switching. Compared with the obtained results [42], [43], [44], [45], [46], [47], [48], there are mainly threefold contributions in this article. Firstly, based on cooperativecompetitive relation, time-delay, Markov switching and pinning control method, a novel leader-following couple group consensus protocol has been designed for a kind of HMASs. Different from literature [42], [43], [44], [45], [46], [47], [48], heterogeneous, cooperative-competitive relation and Markov switching are all considered, which make our model be more related to real situations. Secondly, stochastic delta operator, stability and graph theory are used to obtain sufficient conditions for the couple-group of this system. The obtained results show that LMIs (linear matrix inequalities) can be used to solve the problem of couple-group consensus. Thirdly, according to the achieved sufficient conditions, the balance of in or out degree, strong connectivity and containing a spanning tree of the topologies of MASs can be longer required.
Some numerical examples will be presented to verify the correctness of these achievements.
The remainder of this article will be given as follows. In section II, problem formulation and preliminaries will be listed. In section III, based on pinning control, stochastic delta operator and cooperative-competitive relation, a few novel protocols have been proposed. And some sufficient conditions have been obtained for the couple-group of a kind of HMASs with Markov switching. Proofs have also been finished for these sufficient conditions. In section IV, some numerical experiments have been given to validate the obtained achievements. Finally, conclusion has been drawn in section V.

II. BASIC KNOWLEDGE AND MODEL DESCRIPTION
In this part, model description and some basic knowledge will be addressed, such as Markov process, transitive probability, graph theory and some lemmas.

A. HOMOGENEOUS MARKOV PROCESS
Let { , F, P} be a given a probability space. is a sample space, F is an algebraic event, and P is a probability measure on F. {σ (t) ∈ S = {1, 2, . . . , s} , t ≥ 0} is a homogeneous Markov process with finite states.
From time t to t + T , the transitive probability distribution = (λ ij ) ∈ R s×s of Markov process {σ (t), t ≥ 0} is defined as follows: T > 0 and lim Sometimes, all elements of transitive rate matrix cannot be completely determined. This is to say, some of them are unknown. For instant, if there are four agents in S, the transitive rate matrix is described as matrix (3).
λ ij and λ ij are the unknown and known elements in matrix (3), respectively.
Remark 1: As we know, if some elements of transitive rate matrix are unknown, the conditions for the couple-group of HMSs will become more difficult to determine. Fortunately, according to formulas (1) and (5), we have λ Hence, the sum of given elements can be used to determine the total of unknown elements. Therefore, it is a chance for us to find some sufficient conditions for couple-group of this system.
C. HMASs In our model, the HMASs is composed of m + n agents. The dynamics of the HMASs are described as differential equations (7) and (8).

Definition 1 [24](Asymptotical Couple-Group Consensus):
HMASs (7) and (8) is called asymptotical Couple-Group Consensus if the following conditions (9) can be satisfied for any initial state: Definition 2 (Asymptotical leader-following Couple-Group Consensus): HMASs (7) and (8) is called leader-following couple-group consensus if the following conditions (10) can be satisfied for any initial state and velocity. (10) x 0k (t) and v 0 (t) are position and velocity of the kth leader respectively and the dynamic equation of leader is same as HMASs (7) and (8). Definition 3 [24] (Delta Operator): The delta operator is defined as q is the shift operator which means qz(t) = z(t + T ), T > 0 is the sampling period, and z(t) denotes the status vector. According to Definition.3, the delta jump operator system is given as equation (12).
σ (t) is the switching signal, and the equivalent form of system (12) is described as form (13).
Remark 2: Based on Definition.3, discrete and continuous systems can be unified by the delta system. When the sampled period is small enough, the jump delta operator system is approach to a continuous system. Otherwise, it is more likely approach to a discrete system. In [24], by choosing an appropriate sampling period, both the continuous and discrete system can be described by a unified delta system. Based on delta operator, some more meaningful results can be found in [40] and [49]. In general, the delta operator system has better performance in dealing with poor-conditioned phenomena including consensus of HMASs. Lemma 1: Let {P σ (t) } t≥0 be a homogeneous Markov process. {σ (t), t ≥ 0} is a finite state homogeneous Markov process, and σ (t) takes values in S = {1, 2, . . . , s}. If the mode of the system at the time t is i(σ (t) = i), P i is the corresponding stochastic matrix. P σ (t+T ) is the stochastic matrix of the system at the time t + T . Obviously, P σ (t+T ) ∈ = {P 1 , P 2 , . . . . . . , P s } is a random variable, and we have Definition 3 (Stochastic Delta Operator): Let V (z(t), i) be a stochastic Lyapunov function, then the stochastic delta operator of V (z(t), i) is defined as following formula (15).
Remark 3: Based on lemma.1 and Definition.3, stochastic delta operator can be used to deal with some problems of HMASs with Markov switching. Lemma 2 [49]: Let V (z(t)) be a Lyapunov function in the δ-domain. A jump delta operator system (12) is stochastic asymptotic stability if the following conditions (i) and (ii) can be satisfied: Remark 4 [50]: shows that if T converges to zero, then the stochastic delta operator on V (z(t)) can be regarded as a weak infinitesimal operator. Hence, the weak infinitesimal operator is a special situation of the stochastic delta operator. Therefore, some less conservative results can be achieved by using stochastic delta operator approach.
Remark 5: Based on Lemma 2, the stochastic asymptotic stability can be solved by constructing a Lyapunov function. In fact, switching systems is a special kind of jump system. In this article, Lemma 2 will be utilized to study the couple-group consensus of HMASs (7) and (8) with Markov switching.
Lemma 3 [7]: Ifd , 2, · · ·, m + n}, then the couple group consensus of HMASs (7) and (8) can be realized by pinning control. It should be pointed out the agents with zero in-degree must be pinned. Lemma 4: For any positive semi-definite symmetric matrix M , if there exist two positive integers f 1 and f 2 which satisfy f 2 ≥ f 1 ≥ 1, then the following inequality (16) holds:

III. MAIN RESULTS
In this section, the leader-following couple-group consensus of HMASs (7) and (8) with Markov switching will be firstly analyzed in the part A. The couple-group consensus of HMASs with pinning control and Markov switching will be discussed in the part B.
On the basis of cooperative-competitive relation and Markov switching, a novel leader-following couple-group consensus protocol (17) is designed for HMASs (7) and (8) as follows.
x 0k (t) and v 0 (t) have the same meaning as Definition.
Di denote the neighbor sets of the agent i in the same or different group, respectively. The positive constant α σ (t) and β σ (t) are the coordinated coefficients. a is the element of adjacency matrix A σ (t) . All these parameters may be varying with the change of topology. ι(γ ) is the time-varying function γ m , γ and γ M are given positive integers, and γ m ≤ γ ≤ γ M . The corresponding adjacent matrix A σ (t) of heterogeneous HMASs (7) and (8) can be written as follows: The adjacent matrix A σ (t) is composed of four blocks. The first block A sf denotes the adjacent matrix from second order agent to first order agent, and the third part A σ (t) fs denotes the adjacent matrix from first order agent to second order agent.
Accordingly, the Laplacian matrix L of the HMASs (7) and (8) can be written as matrix form (19).
fij ] ∈ R m×m denote the Laplace matrices of second-order and first-order agents, respectively.
denote the in-degree matrices for different order agents. Moreover, for later analysis, some new matrices are given as follows.
Remark 6: In protocol (17), x i + x j is used to represent the competitive relation between agent i and agent j. They belong to the different groups. And x i − x j is used to represent the cooperative relation between agent i and j agents, they belong to the same group. Based on literature [42], [43], [44], [45], [46], [47], [48], time-delay, cooperative-competitive relation and Markov switching are all considered in protocol (17), which make our model be more approach to real situations.
A. THE LEADER-FOLLOWING COUPLE-GROUP CONSENSUS OF HMASs (7) AND (8) WITH MARKOV SWITCHING The following Theorem.1 gives a sufficient condition for the leader-following couple-group of HMASs (7) and (8). 20) is globally asymptotically stable, then the leader-following couple-group consensus of heterogeneous multi-agent systems (7) and (8) can be realized under the control protocol (17), wherê and Similarly, when i ∈ ρ 2 , we also havė (23) and (24), one has as in (25), shown at the bottom of the page.

based on formulas
Let and, as shown in the equation at the bottom of the page, then we obtainκ and It is obvious that the leader-following couple-group consensus of heterogeneous multi-agent systems (7) and (8) can be realized under the protocol (17) if system (26) is globally asymptotically stable.
Remark 7: According to Theorem.1, a relation has been established between the couple-group consensus of HMASs (7)-(8) and a linear switched system (26). In the next work, delta operator will be introduced to discuss the couplegroup consensus of HMASs (7)- (8). Based on Definition.3, we have Let σ (t) = i ∈ S, then differential equations (26) can be rewritten as follows. where Remark 8: By using traditional shift operation on system (26), we have where and h denotes the sampling period. When h → 0, we can easily obtain In this situation, traditional shift operator cannot fully transform the continuous-time system into a discrete-time system in the z-domain. Especially, when the sampling period h tends to 0, there will be a huge deviation between the original system and the converted system. Based on (31), delta operator method can well make up this defect.
The following Theorem.2 will give a sufficient condition for the couple group of HMASs (7) and (8) with Markov switching if all the transitive probability rates are known.
Theorem 2: If transitive probability rates in Markov switching process are fully known, and there exist symmetric and positive matrices P i > 0, Q > 0, R > 0 satisfying the linear matrix inequality as in (32), shown at the bottom of the next page. then the couple-group consensus of HMAS (7) and (8) with Markov switching can be realized. Proof: Construct a multiple stochastic Lyapunov-Krasovskii function V (κ, t, i) in delta domain as follows. where where Taking the stochastic delta operator manipulations of V (κ, t, i) and using Lemma 1, we have Furthermore, equation (39) is always true for any positive definite real matrix P i .
According to (35)-(39), we have Here as in (41), shown at the bottom of the page. If inequality (41) is true, then we have ϑV (κ, t, i) ≤ 0. Therefore, system (20) is globally asymptotically stable. Based on Theorem 1, the couple-group consensus of HMASs (7) and (8) can be realized under protocol (17).The proof of Theorem 2 is finished. Remark 9: In Theorem.2, the transitive probability rates are known, in this case, the stochastic delta operator can be directly used to deal with the Lyapunov function. However, in many real situations, the transitive probability rates cannot be fully known at any time. Therefore, it is necessary for us to investigate the couple consensus of HMAS (7) and (8) (7) and (8) with Markov switching can be obtained.
Sinceλ ij (∀i ∈ I (i) ukn ) is the unknown transition probability rate, then we have Hence, formula (47) can be further rewritten as following form (48).
Therefore, one has Here, ζ T (t) = δ T κ(t) κ T (t) κ T (t − ι(γ ) and as in (51) k , all elements are known in ith-row, then we only need consider this case:λ ii < −λ (i) k . The multiple stochastic Lyapunov-Krasovskii function is constructed as same as case (a), and Therefore, we obtain Let λ (i) d be the lower limit ofλ ii , then one haŝ In this case,λ ii can be described as a convex linear combination form: . Then if inequalities (56) and (57) are true, one has, as shown in the equation at the bottom of the page, and and = ϑV 1 (κ, t, σ (t)) + ϑV 2 (κ, t) + ϑV 3 (κ, t) + ϑV 4 (κ, t) Based on Case (a), Case (b) and Theorem.2, the proof of Theorem.3 has been completed. Remark 10: According to Theorem.1-Theorem.3, the couple-group consensus of HMASs (7) and (8) with Markov switching can be realized only if some linear matrix inequalities are true. Comparing with existing results, there is no special demand of the topology of the HMASs, such as strong connectivity, containing a spanning tree or the equilibrium of degree. Furthermore, a similar result has also been obtained even if the transitive probability rates in Markov switching are not fully known.

B. COUPLE-GROUP CONSENSUS OF HMASs WITH PINNING CONTROL AND MARKOV SWITCHING
In this section, pinning control will be used to discuss the couple group consensus of HMASs (7) and (8) with Markov switching. Based on the cooperative-competitive relation and Markov switching, a new pinning couple group consensus protocol is designed form as in (59), shown at the bottom of the page. x i is the default point and if d > 0, then the node i will be pinned, otherwise d The following Theorem.4 gives a sufficient condition for the couple-group consensus of HMASs (7) and (8) with Markov switching under pinning controller (59).
Theorem 4: If the following linear switched system (60) is global asymptotic stabilization, then the couple group consensus of HMASs (7) and (8) with Markov switching under pinning controller (59) can be realized, wherê then the following two cases (a) and (b) will be discussed.
According to case (a) and (b), the linear switched differential equation (60) will be obtained.
Theorem 5: For all i, j ∈ S, i = j, λ ij ≥ 0, and if there exist symmetric and positive matrices P i > 0, Q > 0 and R > 0 satisfying the following linear matrix inequality (63).
then the couple-group consensus of HMAS (7) and (8) with Markov switching will be realized under the pinning controller (59).
The remainder of proof is similar to Theorem.3, and it is omitted.
Remark 11: In this paper, Markov switching, cooperativecompetitive relation and time delay are all considered. Different from literature [11], [13], [52], [53], [54], [55], [56], [57], our models are more related to real situations. For instance, the dynamics of agents are homogeneous in [11]. In [13], the topology structure of MASs is supposed to be fixed. In [52], the topology structure of MASs is supposed to be a special fixed switching situation. In [53], the topology structure of MASs is supposed to satisfy the condition of in-degree balance. In [54], [55], [56], and [57], the relations between agents are supposed to be cooperative or competitive. In [58], a second-order multiple agents are used to finish optimal persistent monitoring tasks. In [59], the leaderfollowing consensus problem for a class of identical nonlinear time delay multiagent systems is studied. Paper [60] investigates the output formation-containment problem of interacted heterogeneous linear systems with leader or the follower. The discrete-time communication manner is applied to reduce the communication consumption. Based on the compensator, some algorithms are given to deal with the leader-following output consensus problem for a class of nonlinear time delay multiagent systems in [61]. Therefore, the obtained results in this paper have extended the given achievements in [11], [13], [52], [53], [54], [55], [56], [57], [58], [59], [60], and [61]. On the other hand, it is not difficult for us to find that all the sufficient conditions in the Theorem.1-Theorem.5 can be converted into solving the related LMIs. As we know, LMI is a very effective tool in dealing with some computing problems on linear matrix inequalities. Hence, the obtained results give us some very effective methods in dealing with the problem of couple-group consensus for HMASs.

IV. SOME NUMERICAL EXAMPLES
In this section, some numerical examples will be given to verify the validity of the obtained results.

Example.2:
In this example, the Markov switching network is composed of three sub topologies as Fig.3. In Fig.3.(a) −Fig.3.(c), there is no path from 1 to 4 and there is no path from 4 to 1. Therefore, there is no spanning tree in Fig.3.(a) −Fig.3.(c). Similar with Example 1, the adjacent matrices A 1 , A 2 and A 3 of the three sub topologies are described as follows.
By using Matlab LMI toolbox, a feasible solution forP i ,Q,S andR has been obtained, as shown in the equations at the bottom of the next page. Fig.4 has described the position and velocity trajectories of agents in HMASs (7) and (8) under pinning control (59). And Fig.4.also shows the validity of Theorem.4.
Example 3: In this example, the topology of HMASs (7) and (8) is composed of three parts as Fig.5. Obviously, there is no any spanning tree in Fig.5(a)-Fig.5(c). In Fig.5(a), the out degrees of node 1and node 5 are zero. Both the in degrees of node 1 and node 5 are two. This is to say, Fig.5(a) cannot satisfy the balance of degree. And in pinning control protocol (59), neither of node 1 and node 5 is pinned. Other parameters are completely set as Example 2. Fig.6 shows that the couple group consensus of HMASs (7) and (8) cannot be achieved if the node 1 and node 5 are not pinned.   (7) and (8). Remark 12: In Example.1, there exists a spanning tree in each subgraph, but each sub-topology cannot satisfy the balance of degree and they are not also strong connectivity. In Example.2, there is no any spanning tree in each subgraph. Compared with the obtained results, the demand of topology can be relaxed in this paper. Example.3 shows that the nodes with zero degree must be pinned. Otherwise, the couple-group of HMASs (7) and (8) will not be realized.

V. CONCLUSION
Based on signed digraphs, paper [62] has studied the tracking control problem for second-order MASs with communication constraint scenarios in real engineering applications on tracking dynamics, including external disturbances, asynchronous clocks, and switching interactive topologies. In [63], a novel distributed control scheme is designed for the Cucker-Smale model to examine the leader-follower flocking behavior in cooperative-competitive networks. In [64], event-triggered control is used to investigate the synchronization for a class of switched uncertain neural networks (NNs) with frequent asynchronism. In [65], leader-following consensus for a class of high-order nonlinear multi-agent systems has been studied. Some non-lipschitz continuous control laws have been proposed to realize the finite-time consensus of high-order uncertain nonlinear multi-agent systems. Compared with [58], [59], [60], [61], [62], [63], [64], and [65], leader-following and pinning control couple-group consensus of a kind of HMASs with Markov switching have been studied in this paper, especially cooperative-competitive relation, heterogeneous and Markov switching are taken into account. Based on cooperative and competitive relation and Markov switching, some novel couple group consensus protocols have been designed for this HMASs. By using graph algebra theory, matrix theory and stability theory, the couple group consensus problem of HMASs with Markov switching can be converted into analyzing the stability of a new delta system. Stochastic delta operator, Lyapunov-krasovskii function and LMI (Linear Matrix Inequality) skills have been used to obtain a few sufficient conditions for couple group consensus of these HMASs with Markov switching. In the end, some examples have been presented to address the validity of our achievements. In our future work, event-triggered consensus of HMASs with Markov switching will be considered.