Couple-Group Consensus of Heterogeneous Multi-Agents Systems With Markov Switching and Cooperative-Competitive Interaction

In this paper, couple-group consensus is investigated for a kind of heterogeneous multi-agent systems (HMASs) with Markov switching. Some novel couple-group consensuses have been proposed, in which cooperative-competitive interaction, Markov switching and time delay are all considered. For Markov switching, the transitive rate of probability can be divided into two cases: known or partly known. Based on stochastic delta operator, probability, graph and stability theories, the leader-following and pinning couple-group consensus of this system be converted into analyzing the stability of related switching delta operator system. In the obtained results, some conservative conditions, such as the balance of in or out degree, strong connectivity and containing a spanning tree, can be longer strictly demanded. Some numerical examples are given to show the validity of the acquired results.

SECTION I. Introduction

Recent years, consensus has become a hot issue in distributed control of MASs due its promising applications in many domains, such as the formation of mobile vehicles [1], [2], distributed control in sensor networks [3], [4], group decision-making systems [5] and so forth. The mainly ideal of consensus is to design an effective protocol to make all agents reach a same state. As an extension of consensus, group (cluster) consensus means that agents in the whole system can be divided into several groups. Agents in different groups will reach different states. Agents in the same group will reach same state. In some extent, consensus or synchronization, is a special case of group consensus. In fact, group consensus is very common in real life [5], [6], [7], [8], [9], [11], [12]. Hence, it is important for us to investigate the group consensus of MASs. So far, a lot of meaningful achievements have been acquired for the group consensus of MASs [6], [7], [8], [9], [10], [11], [12]. In [6], pinning control is used to realize group tracking consensus of 2nd-order nonlinear MASs. In [7], based on cooperative-competitive relation, some sufficient conditions have been obtained for the group consensus of a kind of HMASs. In [8] and [9], group consensus of MASs with switching topology and time delay has been discussed. In [10], distributed PID controller is used to discuss the consensus of first-order MASs with time delay. In [11] and [12], group consensus of 2nd-order or even high-order MASs have been studied respectively. It worth pointing out that most of obtained results are needed to suppose the topologies of MASs are homogeneous. In a homogeneous MASs, all agents have the same dynamics. However, in many practical cases, it is impossible for each agent to have the identical dynamics. Hence, it is essential for us to investigate the consensus of heterogeneous MASs [13], [14], [15], [16], [17]. In [13], Sun and Zhang have focused on the output sign consensus problem of HMASs with fixed or switching signed graphs. In [14], time-varying output formation tracking problem of heterogeneous linear MASs has been concerned. More correlative results can be found in [15], [16], and [17]. On the other hand, from the view of control cost, it is quite difficult for us to control all nodes to achieve consensus in many large-scale complex networks. For this purpose, the pinning control method has been proposed. According to existing results, the pinning control can be used to realize the consensus or synchronization of MASs [18], [19]. In [18], event-triggered strategy and pinning control have been used to achieve consensus of MASs. In [19], predictive pinning control has been applied to obtain the consensus of MASs with time delay. In [20], pinning control is applied for the leader-following synchronization of MASs. Different from other control protocols, it is not necessary for us to control all nodes in pinning control strategy. The obtained results have also shown that the pinning control scheme can reduce the control cost. Furthermore, most of the topology structures of MASs are supposed to be fixed when we discuss consensus of these systems [21], [22]. Nevertheless, in most practical cases, the topology of MASs may change with time varying, it is not fixed [23], [24], [25]. Thus, it is also more significant for us to investigate the consensus of MASs switching topology [23], [24], [25], [26]. However, among most existing results, the switching ways are usually assumed to obey some specific rules such as ADT or MDADT [24], [26]. In [24], MDADT method has been used to address the consensus problem of HMASs with switching topology. In [26], based on ADT, switching scheme has been proposed for compensating packet dropout. In fact, various internal and external disturbances may exist in real system, such as uncertain distance, time delay, connection failure and so on. All these reasons can lead to different switching ways. Sometimes, these switching ways are even stochastic. In this situation, it is very difficult for us to use the novel mathematical models or theories to analyze the performance of these systems. How to describe these various switching manners? How to analyze the couple-group of these systems with switching, especially stochastic switching manner is considered? It is very necessary for us to establish new mathematic model and introduce novel analytic tool. Fortunately, Markov process has excellent merit in describing uncertain abrupt phenomena. It has been widely used in robust detection filters, image recognition, and other fields [27], [28], [29]. Therefore, it is natural for us to introduce the Markov process to describe MASs with stochastic switching manner. Of course, the transitive probability matrix of Markov process may not be fully known in many situations. It is very significant for us to analyze the consensus of MASs under Markov switching with part known transitive probability. In this paper, two cases of Markov process with known or unknown transitive rate matrix will be discussed when we investigate the couple-group of HMASs.

Moreover, most of the acquired achievements of MASs must obey the assumption that relations among agents are cooperative or competitive [30], [31], [32], [33]. However, cooperative and competitive relation often coexist in natural world. Therefore, it is essential for us to study the related problems of MASs with cooperative and competitive relations [7], [24]. As we know, delta operator is an effective tool in analyzing the performance of a system whatever it is discrete or continuous [34], [35], [36], [37], [38], [39]. In [37], the delta operator has been firstly proposed to unify the discrete and continuous system into a single framework. In [38], the delta operator has been used to detect observer-based fault. In [39], the delta operator has been applied to analyze sliding mode control. Undoubtedly, based on delta operator, more and more meaningful results have been derived [27], [40]. According to existing literature, the delta operator can also be applied to study the consensus problem of MASs [24], [41]. In [24], the delta operator has been used to handle the group consensus problem of HMASs in cooperative- competitive networks. And in [41], the delta operator has been utilized to discuss the consensus problem of MASs with faults and mismatches. Inspired by the above discussions, this paper will use pinning scheme and stochastic delta operator to investigate the couple group consensus of HMASs with cooperative-competitive relation and Markov switching. Compared with the obtained results [42], [43], [44], [45], [46], [47], [48], there are mainly threefold contributions in this article. Firstly, based on cooperative-competitive relation, time-delay, Markov switching and pinning control method, a novel leader-following couple group consensus protocol has been designed for a kind of HMASs. Different from literature [42], [43], [44], [45], [46], [47], [48], heterogeneous, cooperative-competitive relation and Markov switching are all considered, which make our model be more related to real situations. Secondly, stochastic delta operator, stability and graph theory are used to obtain sufficient conditions for the couple-group of this system. The obtained results show that LMIs (linear matrix inequalities) can be used to solve the problem of couple-group consensus. Thirdly, according to the achieved sufficient conditions, the balance of in or out degree, strong connectivity and containing a spanning tree of the topologies of MASs can be longer required. Some numerical examples will be presented to verify the correctness of these achievements.

The remainder of this article will be given as follows. In section II, problem formulation and preliminaries will be listed. In section III, based on pinning control, stochastic delta operator and cooperative-competitive relation, a few novel protocols have been proposed. And some sufficient conditions have been obtained for the couple-group of a kind of HMASs with Markov switching. Proofs have also been finished for these sufficient conditions. In section IV, some numerical experiments have been given to validate the obtained achievements. Finally, conclusion has been drawn in section V.

SECTION II. Basic Knowledge and Model Description

In this part, model description and some basic knowledge will be addressed, such as Markov process, transitive probability, graph theory and some lemmas.

A. Homogeneous Markov Process

Let $\{\Omega,F,P\}$ be a given a probability space. $\Omega $ is a sample space, $F$ is an algebraic event, and $P$ is a probability measure on $F$ . $\left \{{{\sigma (t)\in S=\left \{{{1,2,\ldots,s} }\right \},t\ge 0} }\right \}$ is a homogeneous Markov process with finite states.

From time $t$ to $t+T$ , the transitive probability distribution $\Lambda =(\lambda _{ij})\in R^{s\times s}$ of Markov process $\left \{{{\sigma (t),t\ge 0} }\right \}$ is defined as follows:\begin{align*} \Pr (\sigma (t_{k} +T)=&j\vert \sigma (t_{k})=i) \\=&\begin{cases} \displaystyle \lambda _{ij} T+o(T),& i\ne j \\ \displaystyle 1+\lambda _{ij} T+o(T),& i=j \end{cases}\tag{1}\end{align*} $T>0$ and $\lim \limits _{T\to 0} \frac {o(T)}{T}=0$ . For all $i,j\in S,i\ne j$ , let $\lambda _{ij} >0$ be the transition rate from topology $i$ at time $t$ to topology $j$ at time $t+T$ , and $\lambda _{ii} =-\sum \limits _{j=1,j\ne i}^{s} {\lambda _{ij}},T>0$ . The transitive rate matrix $\Sigma _{\sigma } $ is defined as matrix form (2).\begin{align*} \Sigma _{\sigma } =\left [{ {{\begin{array}{cccccccccccccccccccc} {\lambda _{11}} &\quad {\lambda _{12}} &\quad \cdots &\quad {\lambda _{1s}} \\ {\lambda _{21}} &\quad {\lambda _{22}} &\quad \cdots &\quad {\lambda _{2s}} \\ \vdots &\quad \vdots &\quad \ddots &\quad \vdots \\ {\lambda _{s1}} &\quad {\lambda _{s2}} &\quad \cdots &\quad {\lambda _{ss}} \\ \end{array}}} }\right]\tag{2}\end{align*} Sometimes, all elements of transitive rate matrix cannot be completely determined. This is to say, some of them are unknown. For instant, if there are four agents in $S$ , the transitive rate matrix is described as matrix (3).\begin{align*} \Sigma _{\sigma } =\left [{ {{\begin{array}{cccccccccccccccccccc} {\hat {\lambda }_{11}} &\quad {\lambda _{12}} & \quad {\hat {\lambda }_{13}} & \quad {\lambda _{14}} \\ {\lambda _{21}} &\quad {\hat {\lambda }_{22}} &\quad {\lambda _{23} } &\quad {\hat {\lambda }_{24}} \\ {\hat {\lambda }_{31}} &\quad {\lambda _{32}} &\quad {\hat {\lambda }_{33}} &\quad {\hat {\lambda }_{34}} \\ {\hat {\lambda }_{41}} &\quad {\hat {\lambda }_{42}} &\quad {\lambda _{43}} &\quad {\hat {\lambda }_{44}} \\ \end{array}}} }\right]\tag{3}\end{align*} $\hat {\lambda }_{ij} $ and $\lambda _{ij} $ are the unknown and known elements in matrix (3), respectively. $\Phi _{kn}^{(i)} $ and $\Phi _{ukn}^{(i)} $ are used to define two sets for the known and unknown elements in the ith row of $\Sigma _{\sigma } $ .\begin{align*} \begin{cases}\Phi_{k n}^{(i)}=\left\{j \mid \lambda_{i j} \text { is known }\right\} \\\Phi_{u k n}^{(i)}=\left\{j \mid \lambda_{i j} \text { is uknown }\right\}\end{cases}\tag{4}\end{align*} Accordingly, $\lambda _{kn}^{(i)} $ and $\lambda _{ukn}^{(i)} $ are the sum of known and unknown transitive rates in row $i$ , respectively, then we have \begin{equation*} \lambda _{kn}^{(i)} =\sum \limits _{j\in \Phi _{kn}^{i}} {\lambda _{ij}}, \quad \lambda _{ukn}^{(i)} =\sum \limits _{j\in \Phi _{ukn}^{i}} {\hat {\lambda }_{ij}}.\tag{5}\end{equation*}

Remark 1:

As we know, if some elements of transitive rate matrix are unknown, the conditions for the couple-group of HMSs will become more difficult to determine. Fortunately, according to formulas (1) and (5), we have $\lambda _{kn}^{(i)} =\sum \limits _{j\in \Phi _{kn}^{i}} {\lambda _{ij}} =-\lambda _{ukn}^{(i)} =-\sum \limits _{j\in \Phi _{ukn}^{i}} {\hat {\lambda }_{ij}}$ ,

Hence, the sum of given elements can be used to determine the total of unknown elements. Therefore, it is a chance for us to find some sufficient conditions for couple-group of this system.

B. Basic Knowledge in Graph

$G^{\sigma (t)}=(V,E^{\sigma (t)},A^{\sigma (t)})$ is a weighted digraph with $n$ nodes, $V=\{v_{1},v_{2},\ldots,v_{n} \}$ is the vertex set and $E^{\sigma (t)}\subseteq V\times V$ is the edge set. The edge of $G^{\sigma (t)}$ is described as $e_{ij}^{\sigma (t)} =(v_{j},v_{i})\in V\times V$ . The weighted matrix $A^{\sigma (t)}=[a_{ij}^{\sigma (t)}]\in R^{n\times n}$ is the adjacency matrix of $G^{\sigma (t)}$ . $\sigma (t)(t\ge 0)$ is a discrete state homogeneous Markov process with continuous time, and $\sigma (t)\in S=\{1,2,\ldots \ldots,s\}$ . $N_{i} =\{v_{j} \left |{ {v_{j} \in V,(v_{j},v_{i})\in E^{\sigma (t)}} }\right.\}$ is the neighbor set of node $v_{i} $ . $L(G^{\sigma (t)})=[l_{ij}^{\sigma (t)}]\in R^{n\times n}$ is the Laplacian matrix of graph $G^{\sigma (t)}$ and \begin{align*} l_{ij}^{\sigma (t)} =\begin{cases} \displaystyle -a_{ij}^{\sigma (t)},& j\ne i \\[0.3pc] \displaystyle \sum \limits _{j=1,j\ne i}^{n} {a_{ij}}^{\sigma (t)},& j=i \end{cases}\tag{6}\end{align*}

C. HMASs

In our model, the HMASs is composed of $m+n$ agents. The dynamics of the HMASs are described as differential equations (7) and (8).\begin{align*}&\begin{cases} \displaystyle \dot {x}_{i} (t)=v_{i} (t)~\\ \displaystyle \dot {v}_{i} (t)=u_{i} (t), \end{cases}\quad i\in \pi _{1} \tag{7}\\&\dot {x}_{i} (t)=u_{i} (t),\quad i\in \pi _{2}\tag{8}\end{align*} $\pi _{1} =\{1,2,\ldots,n\},\pi _{2} =\{n+1,n+2,\ldots,n+m\}$ , $\pi =\pi _{1} \cup \pi _{2} $ , $x_{i} (t)$ , $u_{i} (t)$ and $v_{i} (t)$ represent the position, velocity and input control of agent $i$ , respectively. In equation (7), the second-order agents belong to group $\pi _{1} $ and the first order-agents belong to group $\pi _{2} $ .

D. Some Definitions and Lemmas

Definition 1[24](Asymptotical Couple-Group Consensus):

HMASs (7) and (8) is called asymptotical Couple-Group Consensus if the following conditions (9) can be satisfied for any initial state:\begin{align*}&(i)~\lim \limits _{t\to \infty } \vert \vert x_{i} (t)-x_{j} (t)\vert \vert =0, i,\quad j\in \pi _{k},\quad k=1,2; \\ &(i)~\lim \limits _{t\to \infty } \vert \vert x_{i} (t)-x_{j} (t)\vert \vert \ne 0,\quad i\in \pi _{1} \wedge j\in \pi _{2}; \\ &(ii)~\lim \limits _{t\to \infty } \vert \vert v_{i} (t)-v_{j} (t)\vert \vert =0,i,\quad j\in \pi _{1}.\tag{9}\end{align*}

Definition 2 (Asymptotical leader-following Couple-Group Consensus):

HMASs (7) and (8) is called leader-following couple-group consensus if the following conditions (10) can be satisfied for any initial state and velocity.\begin{align*}&(i)~\lim \limits _{t\to \infty } \vert \vert x_{i} (t)-x_{0k} (t)\vert \vert =0,\quad i\in \pi _{1} \cup \pi _{2},\quad k=1,2; \\&(ii)~\lim \limits _{t\to \infty } \vert \vert v_{i} (t)-v_{0} (t)\vert \vert =0,\quad i\in \pi _{1}.\tag{10}\end{align*} $x_{0k} (t)$ and $v_{0} (t)$ are position and velocity of the $k$ th leader respectively and the dynamic equation of leader is same as HMASs (7) and (8).

Definition 3[24] (Delta Operator):

The delta operator is defined as \begin{align*} \delta =\begin{cases} \displaystyle \frac {d}{dt},& T=0 \\[0.5pc] \displaystyle \frac {q-1}{T},& T\ne 0 \end{cases}\tag{11}\end{align*} $q$ is the shift operator which means $qz(t)=z(t+T)$ , $T>0$ is the sampling period, and $z(t)$ denotes the status vector. According to Definition.3, the delta jump operator system is given as equation (12).\begin{equation*} \delta z(t)=f_{\sigma (t)} (z(t),z(t_{0})),\quad t\ge t_{0}\tag{12}\end{equation*} $\sigma (t)$ is the switching signal, and the equivalent form of system (12) is described as form (13).\begin{align*} \delta z(t)=\begin{cases} \displaystyle \frac {dz(t)}{dt},& T=0 \\[0.6pc] \displaystyle \frac {z(t+T)-z(t)}{T},& T\ne 0 \end{cases}\tag{13}\end{align*}

Remark 2:

Based on Definition. 3, discrete and continuous systems can be unified by the delta system. When the sampled period is small enough, the jump delta operator system is approach to a continuous system. Otherwise, it is more likely approach to a discrete system. In [24], by choosing an appropriate sampling period, both the continuous and discrete system can be described by a unified delta system. Based on delta operator, some more meaningful results can be found in [40] and [49]. In general, the delta operator system has better performance in dealing with poor-conditioned phenomena including consensus of HMASs.

Lemma 1:

Let $\{P_{\sigma (t)} \}_{t\ge 0} $ be a homogeneous Markov process. $\left \{{{\sigma (t),t\ge 0} }\right \}$ is a finite state homogeneous Markov process, and $\sigma (t)$ takes values in $S=\left \{{{1,2,\ldots,s} }\right \}$ . If the mode of the system at the time $t$ is $i(\sigma (t)=i),P_{i} $ is the corresponding stochastic matrix. $P_{\sigma (t+T)} $ is the stochastic matrix of the system at the time $t+T$ . Obviously, $P_{\sigma (t+T)} \in \Omega =\{P_{1},P_{2},\ldots \ldots,P_{s} \}$ is a random variable, and we have \begin{align*}&\hspace {-2pc} E(P_{\sigma (t+T)}) \\=&Tp_{i1} P_{1} +Tp_{i2} P_{2} +\ldots +(1+Tp_{ii})P_{i} +\ldots +Tp_{is} P_{s} \\=&T\sum \limits _{j=1}^{s} {p_{ij} P_{j} +P_{i}}\tag{14}\end{align*}

Definition 3 (Stochastic Delta Operator):

Let $V(z(t),i)$ be a stochastic Lyapunov function, then the stochastic delta operator of $V(z(t),i)$ is defined as following formula (15).\begin{equation*} \vartheta V(z(t),i)=\frac {1}{T}\left \{{{E[V(z(t+T),j)]-V(z(t))} }\right \}\tag{15}\end{equation*}

Remark 3:

Based on lemma. 1 and Definition. 3, stochastic delta operator can be used to deal with some problems of HMASs with Markov switching.

Lemma 2[49]:

Let $V (z(t))$ be a Lyapunov function in the $\delta $ -domain. A jump delta operator system (12) is stochastic asymptotic stability if the following conditions (i) and (ii) can be satisfied: (i) $V(z(t))\ge 0$ , and $V(z(t))=0$ if and only if $z(t)=0$ ; (ii) $\vartheta V(z(t),i)=\frac {1}{T}\left \{{{E[V(z(t+T),j)]-V(z(t))} }\right \} < 0$ .

Remark 4[50]:

shows that if T converges to zero, then the stochastic delta operator on $V (z(t))$ can be regarded as a weak infinitesimal operator. Hence, the weak infinitesimal operator is a special situation of the stochastic delta operator. Therefore, some less conservative results can be achieved by using stochastic delta operator approach.

Remark 5:

Based on Lemma 2, the stochastic asymptotic stability can be solved by constructing a Lyapunov function. In fact, switching systems is a special kind of jump system. In this article, Lemma 2 will be utilized to study the couple-group consensus of HMASs (7) and (8) with Markov switching.

Lemma 3[7]:

If $\hat {d}_{i}^{\sigma (t)} =0$ and $d_{i}^{\sigma (t)} >0$ , $i\in \left \{{{1,2,\cdot \cdot \cdot,m+n} }\right \}$ , then the couple group consensus of HMASs (7) and (8) can be realized by pinning control. It should be pointed out the agents with zero in- degree must be pinned.

Lemma 4:

For any positive semi-definite symmetric matrix $M$ , if there exist two positive integers $f_{1} $ and $f_{2} $ which satisfy $f_{2} \ge f_{1} \ge 1$ , then the following inequality (16) holds:\begin{align*} \left ({{\sum \limits _{i=f_{1} }^{f_{2}} {z(i)}} }\right)^{T}M\left ({{\sum \limits _{i=f_{1}}^{f_{2}} {z(i)}} }\right)\le (f_{2} -f_{1} +1)\sum \limits _{i=f_{1}}^{f_{2}} {z^{T}(i)Mz(i)} \tag{16}\end{align*}

SECTION III. Main Results

In this section, the leader-following couple-group consensus of HMASs (7) and (8) with Markov switching will be firstly analyzed in the part A. The couple-group consensus of HMASs with pinning control and Markov switching will be discussed in the part B.

On the basis of cooperative-competitive relation and Markov switching, a novel leader-following couple-group consensus protocol (17) is designed for HMASs (7) and (8) as follows.\begin{align*} u_{i} (t)=&\alpha ^{\sigma (t)}\bigg[\sum \limits _{j\in N_{Si}^{\sigma (t)}} {a_{ij}^{\sigma (t)} ((x_{j} (t-\iota (\gamma))-x_{i} (t-\iota (\gamma)))} \\&-\,\sum \limits _{j\in N_{Di}^{\sigma (t)}} {a_{ij}^{\sigma (t)} ((x_{j} (t-\iota (\gamma))+x_{i} (t-\iota (\gamma)))}\bigg] \\&-\,\beta ^{\sigma (t)}v_{i} (t) \\&-\,m_{i} \bigg[\alpha ^{\sigma (t)}(x_{i} (t)-x_{0i} (t))+\beta ^{\sigma (t)}(v_{i} (t)-v_{0} (t)\bigg], \\&\qquad i\in \pi _{1} \\ u_{i} (t)=&\alpha ^{\sigma (t)}\bigg[\sum \limits _{j\in N_{Si}^{\sigma (t)}} {a_{ij}^{\sigma (t)} ((x_{j} (t-\iota (\gamma))-x_{i} (t-\iota (\gamma)))} \\&-\,\sum \limits _{j\in N_{Di}^{\sigma (t)}} {a_{ij}^{\sigma (t)} ((x_{j} (t-\iota (\gamma))+x_{i} (t-\iota (\gamma)))}\bigg] \\&-\,m_{i} \alpha ^{\sigma (t)}(x_{i} (t)-x_{0i} (t)),i\in \pi _{2}\tag{17}\end{align*} $x_{0k} (t)$ and $v_{0} (t)$ have the same meaning as Definition. $2.~N_{Si}^{\sigma (t)} $ and $N_{Di}^{\sigma (t)} $ denote the neighbor sets of the agent $i$ in the same or different group, respectively. The positive constant $\alpha ^{\sigma (t)}$ and $\beta ^{\sigma (t)}$ are the coordinated coefficients. $a_{ij}^{\sigma (t)}$ is the element of adjacency matrix $A^{\sigma (t)}$ . All these parameters may be varying with the change of topology. $\iota (\gamma)$ is the time-varying function and $0\le \iota _{m} \le \iota (\gamma)\le \iota _{M} $ , $\iota _{m} =\gamma _{m} T,\iota (\gamma)=\gamma T$ and $\iota _{M} =\gamma _{M} T$ . $\gamma _{m},\gamma $ and $\gamma _{M} $ are given positive integers, and $\gamma _{m} \le \gamma \le \gamma _{M} $ . The corresponding adjacent matrix $A^{\sigma (t)}$ of heterogeneous HMASs (7) and (8) can be written as follows:\begin{align*} A^{\sigma (t)}=\left [{ {{\begin{array}{cccccccccccccccccccc} {A_{ss}^{\sigma (t)}} &\quad {A_{sf}^{\sigma (t)}} \\ {A_{fs}^{\sigma (t)}} &\quad {A_{ff}^{\sigma (t)}} \\ \end{array}}} }\right]\tag{18}\end{align*} The adjacent matrix $A^{\sigma (t)}$ is composed of four blocks. The first block $A_{s\textrm {s}}^{\sigma (t)} \in R^{n\times n}$ is the adjacent matrix of second-order agents. The fourth part $A_{ff}^{\sigma (t)} \in R^{m\times m}$ is the adjacent matrix of first-order agents. The second part $A_{sf}^{\sigma (t)} $ denotes the adjacent matrix from second order agent to first order agent, and the third part $A_{fs}^{\sigma (t)} $ denotes the adjacent matrix from first order agent to second order agent.

Accordingly, the Laplacian matrix $L$ of the HMASs (7) and (8) can be written as matrix form (19).\begin{align*} L^{\sigma (t)}=D^{\sigma (t)}-A^{\sigma (t)}=\left [{ {{\begin{array}{cccccccccccccccccccc} {L_{ss}^{\sigma (t)} +D_{sf}^{\sigma (t)}} &\quad {-A_{sf}^{\sigma (t)} } \\ {-A_{fs}^{\sigma (t)}} &\quad {L_{ff}^{\sigma (t)} +D_{sf}^{\sigma (t)} } \\ \end{array}}} }\right] \tag{19}\end{align*} $L_{ss}^{\sigma (t)} =[l_{sij}^{\sigma (t)}]\in R^{n\times n}$ and $L_{ff}^{\sigma (t)} =[l_{fij}^{\sigma (t)}]\in R^{m\times m}$ denote the Laplace matrices of second-order and first-order agents, respectively.\begin{align*} D^{\sigma (t)}=&diag\{\hat {d}_{1}^{\sigma (t)},\hat {d}_{2}^{\sigma (t)},\ldots,\hat {d}_{n+m}^{\sigma (t)} \}, \\ D_{sf}^{\sigma (t)}=&diag\left\{{\sum \limits _{j\in N_{i,f}} {a_{ij}^{\sigma (t)} },i\in \rho _{1} }\right\}\end{align*} and \begin{equation*} D_{fs}^{\sigma (t)} =diag\left\{{\sum \limits _{j\in N_{i,s}} {a_{ij}^{\sigma (t)},i\in \rho _{2}} }\right\}\end{equation*} denote the in- degree matrices for different order agents. Moreover, for later analysis, some new matrices are given as follows.\begin{align*} H_{s}^{\sigma (t)}=&\left [{ {h_{sij}^{\sigma (t)}} }\right]\in R^{(n-1)\times (n-1)}, \\ H_{f}^{\sigma (t)}=&\left [{ {h_{fij}^{\sigma (t)}} }\right]\in R^{(m-1)\times (m-1)}\end{align*} and \begin{align*} h_{sij}^{\sigma (t)}=&l_{sij}^{\sigma (t)} -l_{snj}^{\sigma (t)}, \\ h_{fij}^{\sigma (t)}=&l_{fij}^{\sigma (t)} -l_{s(n+m)j}^{\sigma (t)},\quad i\in \pi _{1} \wedge j\in \pi _{2}.\end{align*}

Remark 6:

In protocol (17), $x_{i} +x_{j} $ is used to represent the competitive relation between agent $i$ and agent $j$ . They belong to the different groups. And $x_{i} -x_{j} $ is used to represent the cooperative relation between agent $i$ and $j$ agents, they belong to the same group. Based on literature [42], [43], [44], [45], [46], [47], [48], time-delay, cooperative-competitive relation and Markov switching are all considered in protocol (17), which make our model be more approach to real situations.

A. The Leader-Following Couple-Group Consensus of HMASs (7) and (8) With Markov Switching

The following Theorem.1 gives a sufficient condition for the leader-following couple-group of HMASs (7) and (8). Theorem 1 If system \begin{equation*} \dot {\kappa }(t)=\hat {A}_{\sigma (t)} \kappa (t)+\hat {B}_{\sigma (t)} \kappa (t-\iota (\gamma)),\sigma (t_{0})=\sigma (0)\tag{20}\end{equation*} is globally asymptotically stable, then the leader-following couple-group consensus of heterogeneous multi-agent systems (7) and (8) can be realized under the control protocol (17), where \begin{align*} \hat {A}_{\sigma (t)}=&\left [{ {{\begin{array}{cccccccccccccccccccc} 0 & 0 & {I_{n\times n}} \\ 0 & {-\alpha ^{\sigma (t)}M_{f}^{\sigma (t)}} & 0 \\ {-\alpha ^{\sigma (t)}M_{s}^{\sigma (t)}} & 0 & {-\beta ^{\sigma (t)}\hat {M}_{s}^{\sigma (t)}} \\ \end{array}}} }\right]\tag{21}\\ \hat {B}_{\sigma (t)}=&\left [{ {{\begin{array}{cccccccccccccccccccc} 0 & 0 & 0 \\ {\alpha ^{\sigma (t)}A_{fs}^{\sigma (t)}} & {-\alpha ^{\sigma (t)}(L_{ff}^{\sigma (t)} +D_{fs}^{\sigma (t)})} & 0 \\ {-\alpha ^{\sigma (t)}(L_{ss}^{\sigma (t)} +D_{sf}^{\sigma (t)})} & {\alpha ^{\sigma (t)}A_{sf}^{\sigma (t)}} & 0 \\ \end{array}}} }\right] \\ M_{s}^{\sigma (t)}=&diag[m_{1},\ldots,m_{n}], \\ M_{f}^{\sigma (t)}=&diag[m_{n+1},\ldots,m_{n+m}], \\ \hat {M}_{s}^{\sigma (t)}=&diag[1+m_{1},\ldots,1+m_{n}],\tag{22}\end{align*} and \begin{align*} k(t)=[x_{1} (t)-x_{n} (t),\cdot \cdot \cdot,x_{n-1} (t)-x_{n} (t),x_{n+1} (t)-x_{n+m} (t), \\ \cdot \cdot \cdot,x_{n+m-1} -x_{n+m},v_{1} (t)-v_{n} (t),v_{n-1} (t)-v_{n} (t)]^{T}.\end{align*}

Proof:

Define $\bar {x}_{i} (t)=x_{i} (t)-x_{0i} (t),\bar {v}_{i} (t)=v_{i} (t)-v_{0} (t)$ , $\bar {x}_{s} (t)=[\bar {x}_{1}^{T} (t),\ldots,\bar {x}_{n}^{T} (t)]^{T},\bar {x}_{f} (t)=[\bar {x}_{n+1}^{T} (t),\ldots, \bar {x}_{n+m}^{T} (t)]^{T}$ , and $\bar {v}(t)=[\bar {v}_{1}^{T} (t),\ldots,\bar {v}_{n}^{T} (t)]^{T}$ . When $i\in \rho _{1} $ , according to protocol (17), we have \begin{align*} \bar {v}_{i} (t)=&\alpha ^{\sigma (t)}\Bigg [\sum \limits _{j\in N_{Si}^{\sigma (t)}} {a_{ij}^{\sigma (t)} ((\bar {x}_{j} (t-\iota (\gamma))-\bar {x}_{i} (t-\iota (\gamma)))} \\&-\,\sum \limits _{j\in N_{Di}^{\sigma (t)}} {a_{ij}^{\sigma (t)} ((\bar {x}_{j} (t-\iota (\gamma))+\bar {x}_{i} (t-\iota (\gamma)))}\Bigg ] \\&-\,\beta ^{\sigma (t)}\bar {v}_{i} (t)-m_{i} \Bigg [\alpha ^{\sigma (t)}(\bar {x}_{i} (t)-\bar {x}_{0i} (t)) \\&+\,\beta ^{\sigma (t)}(\bar {v}_{i} (t)-\bar {v}_{0} (t)\Bigg ]\tag{23}\end{align*}

Similarly, when $i\in \rho _{2} $ , we also have \begin{align*} \dot {{\bar {x}}}_{i} (t)=&\alpha ^{\sigma (t)}\Bigg [\sum \limits _{j\in N_{Si}^{\sigma (t)}} {a_{ij}^{\sigma (t)} ((\bar {x}_{j} (t-\iota (\gamma))-x_{i} (t-\iota (\gamma)))} \\&-\,\sum \limits _{j\in N_{Di}^{\sigma (t)}} {a_{ij}^{\sigma (t)} ((x_{j} (t-\iota (\gamma))+\bar {x}_{i} (t-\iota (\gamma)))}\Bigg] \\&-\,m_{i} \alpha ^{\sigma (t)}(\bar {x}_{i} (t)-\bar {x}_{0i} (t)),\quad i\in \rho _{2}\tag{24}\end{align*}

Define $\kappa (t)=[\bar {x}_{s}^{T} (t),\bar {x}_{f}^{T} (t),\bar {v}^{T}(t)]^{T}$ , based on formulas (23) and (24), one has as in (25), shown at the bottom of the page.

Let \begin{align*} \hat {A}_{\sigma (t)} =\left [{ {{\begin{array}{cccccccccccccccccccc} 0 &\quad 0 &\quad {I_{n\times n}} \\ 0 &\quad {-\alpha ^{\sigma (t)}M_{f}^{\sigma (t)}} &\quad 0 \\ {-\alpha ^{\sigma (t)}M_{s}^{\sigma (t)}} &\quad 0 &\quad {-\beta ^{\sigma (t)}\hat {M}_{s}^{\sigma (t)}} \\ \end{array}}} }\right],\end{align*} and, as shown in the equation at the bottom of the page,

then we obtain \begin{equation*} \dot {\kappa }(t)=\hat {A}_{\sigma (t)} \kappa (t)+\hat {B}_{\sigma (t)} \kappa (t-\iota (\gamma))\tag{26}\end{equation*} and \begin{align*} M_{s}^{\sigma (t)}=&diag[m_{1},\ldots,m_{n}],\\ M_{f}^{\sigma (t)}=&diag[m_{n+1},\ldots,m_{n+m}], \\ \hat {M}_{s}^{\sigma (t)}=&diag[1+m_{1},\ldots,1+m_{n}].\end{align*} It is obvious that the leader-following couple-group consensus of heterogeneous multi-agent systems (7) and (8) can be realized under the protocol (17) if system (26) is globally asymptotically stable.

Remark 7:

According to Theorem.1, a relation has been established between the couple-group consensus of HMASs (7)–(8) and a linear switched system (26). In the next work, delta operator will be introduced to discuss the couple-group consensus of HMASs (7)–(8). Based on Definition. 3, we have \begin{align*} \delta \kappa (t)=\frac {\kappa (t+T)-\kappa (t)}{T}=A_{\sigma (t)} \kappa (t)+B_{\sigma (t)} \kappa (t-\iota (\gamma)) \tag{27}\end{align*} Let $\sigma (t)=i\in S$ , then differential equations (26) can be rewritten as follows.\begin{align*} \dot {\kappa }(t)=&\hat {A}_{i} \kappa (t)+\hat {B}_{i} \kappa (t-\iota (\gamma))\tag{28}\\ \delta \kappa (t)=&\frac {\kappa (t+T)-\kappa (t)}{T} \\=&A_{i} \kappa (t)+B_{i} \kappa (t-\iota (\gamma))\tag{29}\end{align*} where \begin{equation*} A_{i} =\frac {e^{\hat {A}_{i} h}-I}{T},\quad B_{i} =\frac {\int _{0}^{T} {e^{\hat {B}_{i} (T-s)}} \hat {B}_{i} ds}{T},\end{equation*} and \begin{align*} \lim \limits _{T\to 0} A_{i}=&\lim \limits _{T\to 0} \frac {e^{\hat {A}_{i} T}-I}{T}=\hat {A}_{i} \\ \lim \limits _{T\to 0} B_{i}=&\lim \limits _{T\to 0} \frac {\int _{0}^{T} {e^{\hat {B}_{i} (T-s)}} \hat {B}_{i} ds}{T}=\hat {B}_{i}\tag{30}\end{align*}

Remark 8:

By using traditional shift operation on system (26), we have \begin{equation*} z(t_{k+1})=A_{z} z(t_{k})+B_{z} z(t_{k} -\iota (\gamma))\tag{31}\end{equation*} where $A_{z} =e^{\hat {A}h},B_{z} =\int _{0}^{h} {e^{\hat {A}(h-s)}} \hat {B}ds$ , and $h$ denotes the sampling period. When $h\to 0$ , we can easily obtain \begin{equation*} \lim \limits _{h\to 0} A_{z} =I, \quad \lim \limits _{h\to 0} B_{z} =O.\end{equation*}

In this situation, traditional shift operator cannot fully transform the continuous-time system into a discrete-time system in the z-domain. Especially, when the sampling period $h$ tends to 0, there will be a huge deviation between the original system and the converted system. Based on (31), delta operator method can well make up this defect.

The following Theorem. 2 will give a sufficient condition for the couple group of HMASs (7) and (8) with Markov switching if all the transitive probability rates are known.

Theorem 2:

If transitive probability rates in Markov switching process are fully known, and there exist symmetric and positive matrices $P_{i} >0,Q>0,R>0$ satisfying the linear matrix inequality as in (32), shown at the bottom of the next page.

then the couple-group consensus of HMAS (7) and (8) with Markov switching can be realized.

Proof:

Construct a multiple stochastic Lyapunov-Krasovskii function $V(\kappa,t,i)$ in delta domain as follows.\begin{align*} V(\kappa,t,i)=V_{1} (\kappa,t,i)+V_{2} (\kappa,t)+V_{3} (\kappa,t)+V_{4} (\kappa,t) \tag{33}\end{align*} where \begin{align*} V_{1} (\kappa,t,i)=&\kappa {}^{T}(t)P_{i} \kappa (t), \\ V_{2} (\kappa,t)=&T\sum \limits _{\gamma _{1} =1}^{\gamma} {\kappa ^{T}} (t-\gamma _{1} T)Q\kappa (t-\gamma _{1} T), \\ V_{3} (\kappa,t)=&T^{2}\sum \limits _{\gamma _{1} =\gamma _{m} +1}^{\gamma _{M}} {\sum \limits _{\gamma _{2} =1}^{\gamma _{1}} {\kappa ^{T}(t-\gamma _{2} T)Q\kappa (t-\gamma _{2} T)}}, \\ V_{4} (\kappa,t)=&\sum \limits _{\gamma _{1} =1}^{\gamma} {\sum \limits _{\gamma _{2} =1}^{\gamma _{1}} {g^{T}(t-\gamma _{2} T)Rg(t-\gamma _{2} T)}}\tag{34}\end{align*} where $g(t-\gamma _{2} T)=\kappa (t-\gamma _{2} T)-\kappa (t-(\gamma _{2} -1)T)$ , $\iota (\gamma)=\gamma T$ , then we have $\delta (\kappa (t))=-g(t)/T$ .

Taking the stochastic delta operator manipulations of $V(\kappa,t,i)$ and using Lemma 1, we have \begin{align*}&\hspace {-1pc} \vartheta V_{1} (\kappa,t,\sigma (t)) \\=&\frac {1}{T}\Bigg \{E[\kappa ^{T}(t+T)P_{\sigma (t+T)} \kappa (t+T)] \\&-\,\kappa ^{T}(t)P_{\sigma (t)} \kappa (t)\Bigg \} \\=&\sum \limits _{j=1}^{s} {\lambda _{ij} \kappa ^{T}(t+T)} P_{j} \kappa (t+T) \\&+\,\frac {1}{T}[\kappa ^{T}(t+T)P_{i} \kappa (t+T)-\kappa ^{T}(t)P_{i} \kappa (t)] \\=&T^{2}\sum \limits _{j=1}^{s} {\lambda _{ij} \delta ^{T}\kappa (t)P_{j} \delta \kappa (t)} +T\sum \limits _{j=1}^{s} {\lambda _{ij} \delta ^{T}\kappa (t)P_{j} \kappa (t)} \\&+\,T\sum \limits _{j=1}^{s} {\lambda _{ij} \kappa ^{T}(t)P_{j} \delta \kappa (t)} +\sum \limits _{j=1}^{s} {\lambda _{ij} \kappa ^{T}(t)P_{j} \kappa (t)} \\&+\,T\delta ^{T}\kappa (t)P_{i} \delta \kappa (t)+\delta ^{T}\kappa (t)P_{i} \kappa (t)+\kappa ^{T}(t)P_{i} \delta \kappa (t) \\=&T^{2}\sum \limits _{j=1}^{s} {\lambda _{ij} \delta ^{T}\kappa (t)P_{j} \delta \kappa (t)} +T\sum \limits _{j=1}^{s} {\lambda _{ij} \delta ^{T}\kappa (t)P_{j} \kappa (t)} \\&+\,T\sum \limits _{j=1}^{s} {\lambda _{ij} \kappa ^{T}(t)P_{j} \delta \kappa (t)} +\sum \limits _{j=1}^{s} {\lambda _{ij} \kappa ^{T}(t)P_{j} \kappa (t)} \\&+\,\kappa ^{T}(t)(A_{i}^{T} P_{i} +P_{i} A_{i})\kappa (t)+\kappa ^{T}(t-\iota (\gamma))B_{i}^{T} P_{i} \kappa (t) \\&+\,\kappa ^{T}(t)P_{i} B_{i} \kappa (t-\iota (\gamma)) \\&+\,T\delta ^{T}\kappa (t)P_{i} \delta \kappa (t) \tag{35}\\&\hspace {-1pc}\vartheta V_{2} (\kappa,t) \\=&\frac {1}{T}\Bigg \{\left[{T\sum \limits _{\gamma _{1} =1}^{\gamma} {\kappa ^{T}(t-(\gamma _{1} -1)T)} Q\kappa (t-(\gamma _{1} -1)T)}\right] \\&-\,T\sum \limits _{\gamma _{1} =1}^{\gamma} {\kappa ^{T}(t-\gamma _{1} T)} Q\kappa (t-\gamma _{1} T)\Bigg \} \\\le&\frac {1}{T}\Bigg[T\sum \limits _{\gamma _{1} =1}^{\gamma _{M}} {\kappa ^{T}(t-(\gamma _{1} -1)T)Q\kappa (t-(\gamma _{1} -1)T)} \\&-\,T\sum \limits _{\gamma _{1} =1}^{\gamma} {\kappa ^{T}(t-iT)} Q\kappa (t-\gamma _{1} T)\Bigg] \\\le&\kappa ^{T}(t)Q\kappa (t)-\kappa ^{T}(t-\iota (\gamma))Q\kappa (t-\iota (\gamma)) \\&+\,T\sum \limits _{\gamma _{1} =\gamma _{m} +1}^{\gamma _{M}} {\kappa ^{T}(t-\gamma _{1} T))Q\kappa (t-\gamma _{1} T)} \tag{36}\\&\hspace {-1pc}\vartheta V_{3} (\kappa,t) \\=&T\sum \limits _{\gamma _{1} =\gamma _{m} +1}^{\gamma _{M}} {\Bigg[\sum \limits _{\gamma _{2} =1}^{\gamma _{1}} {\kappa ^{T}(t-(\gamma _{2} -1)T)Q\kappa (t-(\gamma _{2} -1)T)}} \\&-\,\sum \limits _{\gamma _{2} =1}^{\gamma _{1}} {\kappa ^{T}(t-\gamma _{2} T)Q\kappa (t-\gamma _{2} T)} \\=&T\sum \limits _{\gamma _{1} =\gamma _{m} +1}^{\gamma _{M}} {[\kappa ^{T}(t)Q\kappa (t)-\kappa ^{T}(t-\gamma _{1} T)Q\kappa (t-\gamma _{1} T)\Bigg] } \\=&T(\gamma _{M} -\gamma _{m})\kappa ^{T}(t)Q\kappa (t)-T\kappa ^{T}(t-\gamma _{1} T)Q\kappa (t-\gamma _{1} T) \\=&(\iota _{M} -\iota _{m})\kappa ^{T}(t)Q\kappa (t) \\&-\,T\sum \limits _{\gamma _{1} =\gamma _{m} +1}^{\gamma _{M}} {\kappa ^{T}(t-\gamma _{1} T)Q\kappa (t-\gamma _{1} T)} \tag{37}\\&\hspace {-1pc}\vartheta V_{4} (\kappa,t) \\=&\frac {1}{T}\Bigg[\sum \limits _{\gamma _{1} =1}^{\gamma _{M}} {\sum \limits _{\gamma _{2} =1}^{\gamma _{1}} {g^{T}(t-(\gamma _{2} -1)T)} R} g(t-(\gamma _{2} -1)T) \\[7.5pt]&-\,\sum \limits _{\gamma _{1} =1}^{\gamma _{M}} {\sum \limits _{\gamma _{2} =1}^{\gamma _{1}} {g^{T}(t-\gamma _{2} T)} R} g(t-\gamma _{2} T)\Bigg] \\=&\frac {1}{T}\left[{\sum \limits _{\gamma _{1} =1}^{\gamma _{M}} {g^{T}(t)R} g(t)-\sum \limits _{\gamma _{1} =1}^{\gamma _{M}} {g^{T}(t-\gamma _{1} T)R} g(t-\gamma _{1} T)}\right] \\\le&\frac {\gamma _{M} }{T}g^{T}(t)Rg(t) \\&-\,\frac {1}{\gamma _{M} T}\left[{\sum \limits _{\gamma _{1} =1}^{\gamma _{M}} {g(t-\gamma _{1} T)]^{T}R} \sum \limits _{\gamma _{1} =1}^{\gamma _{M}} {g^{T}(t-\gamma _{1} T)} }\right] \\=&-\frac {1}{d_{M}}[\kappa (t-\iota (\gamma))-\kappa (t)]^{T}R[\kappa (t-\iota (\gamma))-\kappa (t)] \\&+\,d_{M} \delta ^{T}\kappa (t)R\delta \kappa (t)~ \\=&-\frac {1}{d_{M}}\kappa ^{T}(t-\iota (\gamma))R\kappa (t-\iota (\gamma)) \\&+\,\frac {1}{d_{M}}\kappa ^{T}(t-\iota (\gamma))R\kappa (t)+\frac {1}{d_{M}}\kappa ^{T}(t)R\kappa (t-\iota (\gamma)) \\&-\,\frac {1}{d_{M}}\kappa ^{T}(t)R\kappa (t)+d_{M} \delta ^{T}\kappa (t)R\delta \kappa (t)\tag{38}\end{align*} Furthermore, equation (39) is always true for any positive definite real matrix $P_{i} $ .\begin{align*} 0=&-2\delta ^{T}\kappa (t)P_{i} [\delta \kappa (t)-A_{i} \kappa (t)-B_{i} \kappa (t-\iota (\gamma))] \\=&-2\delta ^{T}\kappa (t)P_{i} \delta \kappa (t)+2\delta ^{T}\kappa (t)P_{i} A_{i} \kappa (t) \\&+\,2\delta ^{T}\kappa (t)P_{i} B_{i} \kappa (t-\iota (\gamma))\tag{39}\end{align*} According to (35)–​(39), we have \begin{align*} \vartheta V(\kappa,t,i)=&\vartheta V_{1} (\kappa,t,i)+\vartheta V_{2} (\kappa,t)+\vartheta V_{3} (\kappa,t) \\&+\,\vartheta V_{4} (\kappa,t)\le \zeta ^{T}(t)\sum \zeta (t)\le 0\tag{40}\end{align*} Here as in (41), shown at the bottom of the page.

If inequality (41) is true, then we have $\vartheta V(\kappa,t,i)\le ~0$ . Therefore, system (20) is globally asymptotically stable. Based on Theorem 1, the couple-group consensus of HMASs (7) and (8) can be realized under protocol (17).The proof of Theorem 2 is finished.

Remark 9:

In Theorem. 2, the transitive probability rates are known, in this case, the stochastic delta operator can be directly used to deal with the Lyapunov function. However, in many real situations, the transitive probability rates cannot be fully known at any time. Therefore, it is necessary for us to investigate the couple consensus of HMAS (7) and (8) with part known transitive probability rate in Markov switching. Theorem. 3 gives a sufficient condition for the couple group consensus of this system.

Theorem 3:

For all $i,j\in S,i\ne j,\lambda _{ij} \ge 0$ , and $\lambda _{ii} =\,\,-\sum \limits _{j=1,j\ne i}^{s} {\lambda _{ij}}$ , if there exist symmetric and positive matrices $P_{i} >0,Q>0,R>0$ satisfying the following two linear matrix inequalities as in (42) and (43), shown at the bottom of the page.

Case (a): $\lambda _{ii} $ is known ($i\in \Phi _{kn}^{(i)})$ . Case (b): $\hat {\lambda }_{ii} $ is unknown ($i\in \Phi _{ukn}^{(i)})$ , then the couple group consensus of HMAS (7) and (8) with Markov switching can be obtained.

Proof:

Similar to Theorem. 2, a multiple stochastic Lyapunov-Krasovskii function $V(\kappa,t,i)$ is constructed as following form (44).\begin{align*} V(\kappa,t,i)=V_{1} (\kappa,t,i)+V_{2} (\kappa,t)+V_{3} (\kappa,t)+V_{4} (\kappa,t) \tag{44}\end{align*} And \begin{align*} V_{1} (\kappa,t,\sigma (t)=i)=&\kappa {}^{T}(t)P_{i} \kappa (t), \\ V_{2} (\kappa,t)=&T\sum \limits _{\gamma _{1} =1}^{\gamma} {\kappa ^{T}} (t-\gamma _{1} T)Q\kappa (t-\gamma _{1} T), \\ V_{3} (\kappa,t)=&T^{2}\sum \limits _{\gamma _{1} =\gamma _{m} +1}^{\gamma _{M}} \sum \limits _{\gamma _{2} =1}^{\gamma _{1}} \\&\times \,{\kappa ^{T}(t-\gamma _{2} T)Q\kappa (t-\gamma _{2} T)}, \\ V_{4} (\kappa,t)=&\sum \limits _{\gamma _{1} =1}^{\gamma} {\sum \limits _{\gamma _{2} =1}^{\gamma _{1}} {g^{T}(t-\gamma _{2} T)Rg(t-\gamma _{2} T)}} \tag{45}\end{align*} where $\iota (\gamma)=\gamma T,g(t-\gamma _{2} T)=\kappa (t-\gamma _{2} T)-\kappa (t-(\gamma _{2} -1)T)$ , then $\delta (\kappa (t))=-g(t)/T$ . Next, we will discuss in two situations (a) and (b). Case (a): $\lambda _{ii} $ is known ($i\in \Phi _{kn}^{(i)})$ . Taking the stochastic delta operator manipulation on $V(\kappa,t,i)$ , we have \begin{align*}&\hspace {-1.3pc} \vartheta V_{1} (\kappa,t,\sigma (t)) \\=&\frac {1}{T}\{E[\kappa ^{T}(t+T)P_{\sigma (t+T)} \kappa (t+T)]-\kappa ^{T}(t)P_{\sigma (t)} \kappa (t)\} \\=&\sum \limits _{j=1}^{s} {\lambda _{ij} \kappa ^{T}(t+T)} P_{j} \kappa (t+T)+\frac {1}{T}[\kappa ^{T}(t+T)P_{i} \kappa (t+T) \\&-\,\kappa ^{T}(t)P_{i} \kappa (t)] \\=&T^{2}\sum \limits _{j=1}^{s} {\lambda _{ij} \delta ^{T}\kappa (t)P_{j} \delta \kappa (t)} +T\sum \limits _{j=1}^{s} {\lambda _{ij} \delta ^{T}\kappa (t)P_{j} \kappa (t)} \\&+\,\sum \limits _{j=1}^{s} {\lambda _{ij} \kappa ^{T}(t)P_{j} \delta \kappa (t)} \\&+\,\sum \limits _{j=1}^{s} {\lambda _{ij} \kappa ^{T}(t)P_{j} \kappa (t)} +T\delta ^{T}\kappa (t)P_{i} \delta \kappa (t) \\&+\,\delta ^{T}\kappa (t)P_{i} \kappa (t)+\kappa ^{T}(t)P_{i} \delta \kappa (t)\tag{46}\end{align*} Formula (46) can be rewritten as follows.\begin{align*}&\hspace {-2pc} \vartheta V_{1} (\kappa,t,\sigma (t)) \\=&T^{2}\delta ^{T}\kappa (t)P_{k}^{(i)} \delta \kappa (t)-T^{2}\lambda _{k}^{(i)} \sum \limits _{j\in I_{uk}^{(i)}} {\frac {\hat {\lambda }_{ij} }{-\lambda _{k}^{(i)}}} \delta ^{T}\kappa (t)P_{j} \delta \kappa (t)~ \\&+\,T\delta ^{T}\kappa (t)P_{k}^{(i)} \kappa (t)-T\lambda _{k}^{(i)} \sum \limits _{j\in I_{uk}^{(i)}} {\frac {\hat {\lambda }_{ij} }{-\lambda _{k}^{(i)}}} \delta ^{T}\kappa (t)P_{j} \kappa (t)~ \\&+\,T\kappa ^{T}(t)P_{k}^{(i)} \delta \kappa (t)-T\lambda _{k}^{(i)} \sum \limits _{j\in I_{uk}^{(i)}} {\frac {\hat {\lambda }_{ij}}{-\lambda _{k}^{(i)}}} \kappa ^{T}(t)P_{j} \delta \kappa (t) \\&+\,\kappa ^{T}(t)P_{k}^{(i)} \kappa (t)-\lambda _{k}^{(i)} \sum \limits _{j\in I_{uk}^{(i)}} {\frac {\hat {\lambda }_{ij}}{-\lambda _{k}^{(i)}}} \kappa ^{T}(t)P_{j} \kappa (t)~ \\&+\,\kappa ^{T}(t)(A_{i}^{T} P_{i} +P_{i} A_{i})\kappa (t)+\kappa ^{T}(t-\iota (\gamma))B_{i}^{T} P_{i} \kappa (t) \\&+\,\kappa ^{T}(t)P_{i} B_{i} \kappa (t-\iota (\gamma))+T\delta ^{T}\kappa (t)P_{i} \delta \kappa (t)\tag{47}\end{align*} Since $\hat {\lambda }_{ij} (\forall i\in I_{ukn}^{(i)})$ is the unknown transition probability rate, then we have \begin{equation*} 0\le -\frac {\hat {\lambda }_{ij} }{\lambda _{k}^{(i)}}\le 1~\text {and}~\sum \nolimits _{j\in I_{uk}^{(i)}} {\left({-\frac {\hat {\lambda }_{ij}}{\lambda _{k}^{(i)}}}\right)} =1.\end{equation*} Hence, formula (47) can be further rewritten as following form (48).\begin{align*}&\hspace {-2pc}\vartheta V_{1} (\kappa,t,\sigma (t)) \\=&\sum \limits _{j\in I_{uk}^{(i)}} {\frac {\hat {\lambda }_{ij}}{-\lambda _{k}^{(i)}}} [T^{2}\delta ^{T}\kappa (t)P_{k}^{(i)} \delta \kappa (t)~ \\&-\,T^{2}\lambda _{k}^{(i)} \delta ^{T}\kappa (t)P_{j} \delta \kappa (t) \\&+\,T\delta ^{T}\kappa (t)P_{k}^{(i)} \kappa (t)-T\lambda _{k}^{(i)} \delta ^{T}\kappa (t)P_{j} \kappa (t)~ \\&+\,T\kappa ^{T}(t)P_{k}^{(i)} \delta \kappa (t)-T\lambda _{k}^{(i)} \kappa ^{T}(t)P_{j} \delta \kappa (t)~ \\&+\,\kappa ^{T}(t)P_{k}^{(i)} \kappa (t)-\lambda _{k}^{(i)} \kappa ^{T}(t)P_{j} \kappa (t)~ \\&+\,\kappa ^{T}(t)(A_{i}^{T} P_{i} +P_{i} A_{i})\kappa (t)+\kappa ^{T}(t-\iota (\gamma))B_{i}^{T} P_{i} \kappa (t)~ \\&+\,\kappa ^{T}(t)P_{i} B_{i} \kappa (t-\iota (\gamma))+T\delta ^{T}\kappa (t)P_{i} \delta \kappa (t)]\tag{48}\end{align*} $\vartheta V_{2} (\kappa,t),\vartheta V_{3} (\kappa,t)$ and $\vartheta V_{4} (\kappa,t)$ can be computed as Theorem. 2.

Similarly, for any positive definite real matrix $P_{i} $ , the following equality (49) is always true.\begin{align*} 0=&-2\delta ^{T}\kappa (t)P_{i} [\delta \kappa (t)-A_{i} \kappa (t)-B_{i} \kappa (t-\iota (\gamma))] \\=&-2\delta ^{T}\kappa (t)P_{i} \delta \kappa (t)+2\delta ^{T}\kappa (t)P_{i} A_{i} \kappa (t)~ \\&+\,2\delta ^{T}\kappa (t)P_{i} B_{i} \kappa (t-\iota (\gamma))\tag{49}\end{align*} Therefore, one has \begin{align*}&\hspace {-2pc} \vartheta V(\kappa,t,\sigma (t)) \\=&\vartheta V_{1} (\kappa,t,\sigma (t))+\vartheta V_{2} (\kappa,t)+\vartheta V_{3} (\kappa,t)+\vartheta V_{4} (\kappa,t) \\\le&\zeta ^{T}(t)\Sigma _{0} \zeta (t) < 0\tag{50}\end{align*} Here, $\begin{aligned} \zeta ^{T}(t)=\left [{ {{\begin{array}{cccccccccccccccccccc} {\delta ^{T}\kappa (t)} & {\kappa ^{T}(t)} & {\kappa ^{T}(t-\iota (\gamma)} \\ \end{array}}} }\right] \end{aligned}$ and as in (51) and (52), shown at the bottom of the next page.

The proof of Case (a) is finished. Case (b): $\hat {\lambda }_{ii} $ is unknown ($i\in \Phi _{ukn}^{(i)})$ . In this situation, $\hat {\lambda }_{ii} $ is unknown, $\lambda _{k}^{(i)} \ge 0$ and $\hat {\lambda }_{ii} \le -\lambda _{k}^{(i)} $ . Similarly, when $\hat {\lambda }_{ii} =-\lambda _{k}^{(i)} $ , all elements are known in ith-row, then we only need consider this case: $\hat {\lambda }_{ii} < -\lambda _{k}^{(i)} $ . The multiple stochastic Lyapunov-Krasovskii function is constructed as same as case (a), and \begin{align*}&\hspace {-1pc} \vartheta V_{1} (\kappa,t,\sigma (t)) \\=&\frac {1}{T}\{E[\kappa ^{T}(t+T)P_{\sigma (t+T)} \kappa (t+T)]-\kappa ^{T}(t)P_{\sigma (t)} \kappa (t)\} \\=&\sum \limits _{j=1}^{s} {\lambda _{ij} \kappa ^{T}(t+T)} P_{j} \kappa (t+T)+\frac {1}{T}[\kappa ^{T}(t+T)P_{i} \kappa (t+T) \\&-\,\kappa ^{T}(t)P_{i} \kappa (t)] \\=&T^{2}\sum \limits _{j=1}^{s} {\lambda _{ij} \delta ^{T}\kappa (t)P_{j} \delta \kappa (t)} +T\sum \limits _{j=1}^{s} {\lambda _{ij} \delta ^{T}\kappa (t)P_{j} \kappa (t)} \\&+\,T\sum \limits _{j=1}^{s} {\lambda _{ij} \kappa ^{T}(t)P_{j} \delta \kappa (t)} +\sum \limits _{j=1}^{s} {\lambda _{ij} \kappa ^{T}(t)P_{j} \kappa (t)} \\&+\,T\delta ^{T}\kappa (t)P_{i} \delta \kappa (t) \\&+\,\delta ^{T}\kappa (t)P_{i} \kappa (t)+\kappa ^{T}(t)P_{i} \delta \kappa (t) \\=&T^{2}\delta ^{T}\kappa (t)P_{k}^{(i)} \delta \kappa (t)+T^{2}\delta ^{T}\kappa (t)\hat {\lambda }_{ii} P_{i} \delta \kappa (t) \\&+\,T\delta ^{T}\kappa (t)P_{i} \delta \kappa (t)+T\delta ^{T}\kappa (t)P_{k}^{(i)} \kappa (t)+T\delta ^{T}\kappa (t)\hat {\lambda }_{ii} P_{i} \kappa (t) \\&+\,\delta ^{T}\kappa (t)P_{i} \kappa (t)+\kappa ^{T}(t)\hat {\lambda }_{ii} P_{i} \kappa (t)+T\kappa ^{T}(t)P_{k}^{(i)} \delta \kappa (t) \\&+\,T\kappa ^{T}(t)\hat {\lambda }_{ii} P_{i} \delta \kappa (t)+\kappa ^{T}(t)P_{i} \delta \kappa (t)+\kappa ^{T}(t)P_{k}^{(i)} \kappa (t) \\&+\,T^{2}(-\hat {\lambda }_{ii} -\lambda _{k}^{(i)})\sum \limits _{j\in \Phi _{ukn}^{(i)},i\ne j} {\frac {\hat {\lambda }_{ij}}{-\hat {\lambda }_{ij} -\lambda _{k}^{(i)}}} \delta ^{T}\kappa (t)P_{j} \delta \kappa (t) \\&+\,T(-\hat {\lambda }_{ii} -\lambda _{k}^{(i)})\sum \limits _{j\in \Phi _{ukn}^{(i)},i\ne j} {\frac {\hat {\lambda }_{ij}}{-\hat {\lambda }_{ij} -\lambda _{k}^{(i)}}} \delta ^{T}\kappa (t)P_{j} \kappa (t) \\&+\,T(-\hat {\lambda }_{ii} -\lambda _{k}^{(i)})\sum \limits _{j\in \Phi _{ukn}^{(i)},i\ne j} {\frac {\hat {\lambda }_{ij}}{-\hat {\lambda }_{ij} -\lambda _{k}^{(i)}}} \kappa ^{T}(t)P_{j} \delta \kappa (t) \\&+\,(-\hat {\lambda }_{ii} -\lambda _{k}^{(i)})\sum \limits _{j\in I_{uk}^{(i)},i\ne j} {\frac {\hat {\lambda }_{ij}}{-\hat {\lambda }_{ij} -\lambda _{k}^{(i)}}} \kappa ^{T}(t)P_{j} \kappa (t)\tag{53}\end{align*} where $0\le -\frac {\hat {\lambda }_{ij}}{\hat {\lambda }_{ii} +\lambda _{k}^{(i)}}\le 1$ and $\sum \nolimits _{j\in I_{uk}^{(i)},i\ne j} {\left({-\frac {\hat {\lambda }_{ij}}{\hat {\lambda }_{ii} +\lambda _{k}^{(i)} }}\right)} =1$ . Therefore, we obtain \begin{align*}&\hspace {-1pc}\vartheta V(\kappa,t,\sigma (t)) \\=&\sum \nolimits _{j\in I_{uk}^{(i)},i\ne j} {\frac {\hat {\lambda }_{ij} }{-\hat {\lambda }_{ii} -\lambda _{k}^{(i)}}} [T^{2}\delta ^{T}\kappa (t)P_{k}^{(i)} \delta \kappa (t)~ \\&+\,T^{2}\delta ^{T}\kappa (t)\hat {\lambda }_{ii} P_{i} \delta \kappa (t)+T\delta ^{T}\kappa (t)P_{i} \delta \kappa (t) \\&+\,T\delta ^{T}\kappa (t)P_{k}^{(i)} \kappa (t)+T\delta ^{T}\kappa (t)\hat {\lambda }_{ii} P_{i} \kappa (t)+\delta ^{T}\kappa (t)P_{i} \kappa (t) \\&+\,\kappa ^{T}(t)\hat {\lambda }_{ii} P_{i} \kappa (t)+T\kappa ^{T}(t)P_{k}^{(i)} \delta \kappa (t)+T\kappa ^{T}(t)\hat {\lambda }_{ii} P_{i} \delta \kappa (t)~ \\&+\,\kappa ^{T}(t)P_{i} \delta \kappa (t)+\kappa ^{T}(t)P_{k}^{(i)} \kappa (t) \\&+\,T^{2}(-\hat {\lambda }_{ii} -\lambda _{k}^{(i)})\delta ^{T}\kappa (t)P_{j} \delta \kappa (t)+T(-\hat {\lambda }_{ii} \\&-\,\lambda _{k}^{(i)})\delta ^{T}\kappa (t)P_{j} \kappa (t) \\&+\,T(-\hat {\lambda }_{ii} -\lambda _{k}^{(i)})\kappa ^{T}(t)P_{j} \delta \kappa (t)+(-\hat {\lambda }_{ii} -\lambda _{k}^{(i)})\kappa ^{T}(t)P_{j} \kappa (t) \\&+\,\vartheta V_{2} (\kappa,t)+\vartheta V_{3} (\kappa,t)+\vartheta V_{4} (\kappa,t)]\tag{54}\end{align*} If $\Phi _{0} < 0$ and $0\le (\hat {\lambda }_{ij} /-\hat {\lambda }_{ii} -\lambda _{k}^{(i)})\le 1$ , then we have $\vartheta V(\kappa,t,\sigma (t)) < 0$ , and \begin{align*} \Phi _{0}=&T^{2}\delta ^{T}\kappa (t)P_{k}^{(i)} \delta \kappa (t)+T^{2}\delta ^{T}\kappa (t)\hat {\lambda }_{ii} P_{i} \delta \kappa (t) \\&+\,T\delta ^{T}\kappa (t)P_{i} \delta \kappa (t)~ \\&+\,T\delta ^{T}\kappa (t)P_{k}^{(i)} \kappa (t)+T\delta ^{T}\kappa (t)\hat {\lambda }_{ii} P_{i} \kappa (t) \\&+\,\delta ^{T}\kappa (t)P_{i} \kappa (t)\,\,+\kappa ^{T}(t)\hat {\lambda }_{ii} P_{i} \kappa (t)+T\kappa ^{T}(t)P_{k}^{(i)} \delta \kappa (t) \\&+\,T\kappa ^{T}(t)\hat {\lambda }_{ii} P_{i} \delta \kappa (t)~ \\&+\,\kappa ^{T}(t)P_{i} \delta \kappa (t)+\kappa ^{T}(t)P_{k}^{(i)} \kappa (t) \\&+\,T^{2}(-\hat {\lambda }_{ii} -\lambda _{k}^{(i)})\delta ^{T}\kappa (t)P_{j} \delta \kappa (t)~ \\&+\,T(-\hat {\lambda }_{ii} -\lambda _{k}^{(i)})\delta ^{T}\kappa (t)P_{j} \kappa (t) \\&+\,T(-\hat {\lambda }_{ii} -\lambda _{k}^{(i)})\kappa ^{T}(t)P_{j} \delta \kappa (t) \\&+\,(-\hat {\lambda }_{ii} -\lambda _{k}^{(i)})\kappa ^{T}(t)P_{j} \kappa (t)+\vartheta V_{2} (\kappa,t) \\&+\,\vartheta V_{3} (\kappa,t)+\vartheta V_{4} (\kappa,t)\tag{55}\end{align*}

Let $\lambda _{d}^{(i)} $ be the lower limit of $\hat {\lambda }_{ii} $ , then one has \begin{equation*} \hat {\lambda }_{ii} \in [\lambda _{d}^{(i)},-\lambda _{k}^{(i)} +\varepsilon] (\varepsilon >0).\end{equation*} In this case, $\hat {\lambda }_{ii} $ can be described as a convex linear combination form:\begin{equation*} \hat {\lambda }_{ii} =-\alpha \lambda _{k}^{(i)} +\alpha \varepsilon +(1-\alpha)\lambda _{d}^{(i)},\quad \alpha \in [{0,1}].\end{equation*} Then if inequalities (56) and (57) are true, one has, as shown in the equation at the bottom of the page,

and \begin{align*}&\hspace {-1pc}T^{2}\delta ^{T}\kappa (t)[P_{k}^{(i)} +(\varepsilon -\lambda _{k}^{(i)})P_{i} -\varepsilon P_{j}]\delta \kappa (t) \\ &+\,T\delta ^{T}\kappa (t)[P_{k}^{(i)} +(\varepsilon -\lambda _{k}^{(i)})P_{i} -\varepsilon P_{j}]\kappa (t) \\ &+\,T\kappa ^{T}(t)[P_{k}^{(i)} +(\varepsilon -\lambda _{k}^{(i)})P_{i} -\varepsilon P_{j}]\delta \kappa (t) \\ &+\,\kappa ^{T}(t)[P_{k}^{(i)} +(\varepsilon -\lambda _{k}^{(i)})P_{i} -\varepsilon P_{j}]\kappa (t) +T\delta ^{T}\kappa (t)P_{i} \delta \kappa (t) \\ &+\,\delta ^{T}\kappa (t)P_{i} \kappa (t)+\kappa (t)P_{i} \delta \kappa (t) \\ &+\,\vartheta V_{2} (\kappa,t)+\vartheta V_{3} (\kappa,t)+\vartheta V_{4} (\kappa,t) < 0\tag{56}\end{align*} and \begin{align*}&\hspace {-2pc} T^{2}\delta ^{T}\kappa (t)[P_{k}^{(i)} +\lambda _{d}^{(i)} P_{i} -\lambda _{d}^{(i)} P_{j} -\lambda _{k}^{(i)} P_{j}]\delta \kappa (t)~ \\ &+\,T\delta ^{T}\kappa (t)[P_{k}^{(i)} +\lambda _{d}^{(i)} P_{i} -\lambda _{d}^{(i)} P_{j} -\lambda _{k}^{(i)} P_{jj}]\kappa (t)~ \\ &+\,T\kappa ^{T}(t)[P_{k}^{(i)} +\lambda _{d}^{(i)} P_{i} -\lambda _{d}^{(i)} P_{j} -\lambda _{k}^{(i)} P_{j}]\delta \kappa (t)~ \\ &+\,\kappa ^{T}(t)[P_{k}^{(i)} +\lambda _{d}^{(i)} P_{i} -\lambda _{d}^{(i)} P_{j} -\lambda _{k}^{(i)} P_{j}]\kappa (t)~ \\ &+\,T\delta ^{T}\kappa (t)P_{i} \delta \kappa (t)+\delta ^{T}\kappa (t)P_{i} \kappa (t)+\kappa ^{T}(t)P_{i} \delta \kappa (t) \\ &+\,\vartheta V_{2} (\kappa,t)+\vartheta V_{3} (\kappa,t)+\vartheta V_{4} (\kappa,t) < 0\tag{57}\end{align*} Define $\begin{aligned} \zeta ^{T}(t)=\left [{ {{\begin{array}{cccccccccccccccccccc} {\delta ^{T}\kappa (t)} & {\kappa ^{T}(t)} & {\kappa ^{T}(t-\iota (\gamma)} \\ \end{array}}} }\right] \end{aligned}$ , then we have \begin{align*}&\hspace {-2pc} \vartheta V(\kappa,t,\sigma (t)) \\=&\vartheta V_{1} (\kappa,t,\sigma (t))+\vartheta V_{2} (\kappa,t)+\vartheta V_{3} (\kappa,t)+\vartheta V_{4} (\kappa,t) \\=&\zeta ^{T}\Phi _{0} \zeta \le 0\tag{58}\end{align*} Based on Case (a), Case (b) and Theorem. 2, the proof of Theorem. 3 has been completed.

Remark 10:

According to Theorem.1-Theorem.3, the couple-group consensus of HMASs (7) and (8) with Markov switching can be realized only if some linear matrix inequalities are true. Comparing with existing results, there is no special demand of the topology of the HMASs, such as strong connectivity, containing a spanning tree or the equilibrium of degree. Furthermore, a similar result has also been obtained even if the transitive probability rates in Markov switching are not fully known.

B. Couple-Group Consensus of HMASs With Pinning Control and Markov Switching

In this section, pinning control will be used to discuss the couple group consensus of HMASs (7) and (8) with Markov switching. Based on the cooperative-competitive relation and Markov switching, a new pinning couple group consensus protocol is designed form as in (59), shown at the bottom of the page.

$x_{\varpi i} $ is the default point and if $d_{i}^{\sigma (t)} >0$ , then the node $i$ will be pinned, otherwise $d_{i}^{\sigma (t)} =0$ .

The following Theorem. 4 gives a sufficient condition for the couple-group consensus of HMASs (7) and (8) with Markov switching under pinning controller (59).

Theorem 4:

If the following linear switched system (60) \begin{equation*} \dot {\varepsilon }(t)=\hat {F}_{\sigma (t)} \varepsilon (t)+\hat {G}_{\sigma (t)} \varepsilon (t-l(\gamma)),\varepsilon (t_{0})=\varepsilon (0)\tag{60}\end{equation*} is global asymptotic stabilization, then the couple group consensus of HMASs (7) and (8) with Markov switching under pinning controller (59) can be realized, where \begin{align*} \hat {F}_{\sigma (t)}=&\left [{ {{\begin{array}{cccccccccccccccccccc} 0 & 0 & {I_{n\times n}} \\ 0 & {-\alpha ^{\sigma (t)}D_{f}^{\sigma (t)}} & 0 \\ {-\alpha ^{\sigma (t)}D_{s}^{\sigma (t)}} & 0 & {\beta ^{\sigma (t)}I_{n\times n}} \\ \end{array}}} }\right], \\ \hat {G}_{\sigma (t)}=&\left [{ {{\begin{array}{cccccccccccccccccccc} 0 & 0 & 0 \\ {\alpha ^{\sigma (t)}A_{fs}^{\sigma (t)}} & {-\alpha ^{\sigma (t)}[L_{ff}^{\sigma (t)} +D_{fs}^{\sigma (t)}]} & 0 \\ {-\alpha ^{\sigma (t)}[L_{ss}^{\sigma (t)} +D_{sf}^{\sigma (t)}]} & {\alpha ^{\sigma (t)}A_{sf}^{\sigma (t)}} & 0 \\ \end{array}}} }\right] \\ D_{s}=&diag\{d_{1}^{\sigma (t)},d_{2}^{\sigma (t)},\cdot \cdot \cdot,d_{n}^{\sigma (t)} \}, \\ D_{s}=&diag\{d_{n+1}^{\sigma (t)},d_{n+2}^{\sigma (t)},\cdot \cdot \cdot,d_{n+m}^{\sigma (t)} \}.\end{align*}

Proof:

Define $\varepsilon _{i} (t)=x_{i} (t)-x_{\bar {\omega }i},i=1,2,\cdot \cdot \cdot,n+m$ ,\begin{align*} \varepsilon _{s} (t)=&[\varepsilon _{1} (t),\varepsilon _{2} (t),\cdot \cdot \cdot,\varepsilon _{n} (t)]^{T},\\ \varepsilon _{f} (t)=&[\varepsilon _{n+1} (t),\varepsilon _{n+2} (t),\cdot \cdot \cdot,\varepsilon _{n+m} (t)]^{T}, \\ v(t)=&[v_{1} (t),v_{2} (t),\cdot \cdot \cdot,v_{n} (t)]^{T},\\ \varepsilon (t)=&[\varepsilon _{s} (t),\varepsilon _{f} (t),v(t)]^{T}.\end{align*} then the following two cases (a) and (b) will be discussed.

• Case (a).

  $i\in \pi _{1}$ , one has \begin{align*} \dot {v}(t)=&[u_{1} (t),u_{2} (t),\cdot \cdot \cdot,u_{n} (t)]^{T} \\=&-\alpha ^{\sigma (t)}[(L_{ss}^{\sigma (t)} +D_{sf}^{\sigma (t)})\varepsilon _{s} (t-l(\gamma)) \\&-\,A_{sf} \varepsilon _{f} (t-l(\gamma))] \\&-\,\alpha ^{\sigma (t)}D_{s}^{\sigma (t)} \varepsilon _{s} (t)-\beta ^{\sigma (t)}v(t). \\ \dot {\varepsilon }_{f} (t)=&[\dot {\varepsilon }_{1} (t),\dot {\varepsilon }_{2} (t),\cdot \cdot \cdot,\dot {\varepsilon }_{n} (t)]^{T} \\=&[v_{1} (t),v_{2} (t),\cdot \cdot \cdot,v_{n} (t)]^{T}=v(t)\tag{61}\end{align*}

• Case (b).

  $i\in \pi _{2}$ , we have \begin{align*} \dot {\varepsilon }_{f} (t)=&[\dot {\varepsilon }_{n+1} (t),\dot {\varepsilon }_{n+2} (t),\cdot \cdot \cdot,\dot {\varepsilon }_{n+m} (t)]^{T} \\=&[u_{n+1} (t),u_{n+2} (t),\cdot \cdot \cdot,u_{n+m} (t)]^{T} \\=&-\alpha ^{\sigma (t)}[-A_{fs}^{\sigma (t)} \varepsilon _{s} (t-l(\gamma)) \\&+\,(L_{ss}^{\sigma (t)} +D_{fs}^{\sigma (t)})\varepsilon _{f} (t-l(\gamma))] \\&-\,\alpha ^{\sigma (t)}D_{f}^{\sigma (t)} \varepsilon _{f} (f)\tag{62}\end{align*}

According to case (a) and (b), the linear switched differential equation (60) will be obtained.

Based on Theorem.1 and pinning controller (59), the couple-group consensus of HMASs (7) and (8) with Markov switching under pinning controller (59) can be realized.

Theorem 5:

For all $i,j\in S,i\ne j,\lambda _{ij} \ge 0$ , and \begin{equation*} \lambda _{ii} =-\sum \limits _{j=1,j\ne i}^{s} {\lambda _{ij}},\end{equation*} if there exist symmetric and positive matrices \begin{equation*} P_{i} >0,Q>0~\text {and}~R>0\end{equation*} satisfying the following linear matrix inequality (63).\begin{align*} \Omega=&\left [{ {{\begin{array}{cccccccccccccccccccc} {\Omega (1,1)} &\quad {\Omega (1,2)} &\quad {\tilde {P}_{i} G_{i}} &\quad 0 \\ \ast & {\Omega (2,2)} & {\tilde {P}_{i} G_{i}} &\quad {\frac {1}{l_{M}}\tilde {R}} \\ \ast &\quad \ast &\quad {-\tilde {Q}} & 0 \\ \ast &\quad \ast &\quad \ast &\quad {-\tilde {S}-\frac {1}{l_{M} }\tilde {R}} \\ \end{array}}} }\right] < 0 \\ \Omega (1,1)=&T^{2}\sum \limits _{j=1}^{s} {\lambda _{ij} \tilde {P}_{j}} +(T-2)\tilde {P}_{i} +l_{M} \tilde {R}, \\ \Omega (1,2)=&T\sum \limits _{j=1}^{s} {\lambda _{ij} \tilde {P}_{j}} +\tilde {P}_{i} F_{i}, \\ \Omega (2,2)=&\sum \limits _{j=1}^{s} {\lambda _{ij} \tilde {P}_{j}} +F_{i}^{T} \tilde {P}_{i} +\tilde {P}_{i} F_{i} \\&+\,(l_{M} -l_{m} +T+1)\tilde {Q}+\tilde {S}-\frac {1}{l_{M} }\tilde {R},\tag{63}\end{align*} then the couple-group consensus of HMAS (7) and (8) with Markov switching will be realized under the pinning controller (59).

Proof:

A multiple stochastic Lyapunov-Krasovskii function $V(\kappa,t,i)$ is constructed as following form (64).\begin{align*} V(\varepsilon (t),\sigma (t),t)=&V_{1} (\varepsilon (t),\sigma (t),t)+V_{2} (\varepsilon (t),t)+V_{3} (\varepsilon (t),t) \\&+\,V_{4} (\varepsilon (t),t)+V_{5} (\varepsilon (t),t)\end{align*} where \begin{align*} V_{1} (\varepsilon (t),\sigma (t),t)=&\varepsilon {}^{T}(t)\tilde {P}_{\sigma (t)} \varepsilon (t), \\ V_{2} (\varepsilon (t),t)=&T\sum \limits _{\gamma _{1} =1}^{\gamma} {\varepsilon ^{T}} (t-\gamma _{1} T)\tilde {Q}\varepsilon (t-\gamma _{1} T), \\ V_{3} (\varepsilon (t),t)=&T\sum \limits _{\gamma _{1} =1}^{\gamma _{M}} {\varepsilon ^{T}} (t-\gamma _{1} T)\tilde {S}\varepsilon (t-\gamma _{1} T), \\ V_{4} (\varepsilon (t),t)=&T^{2}\sum \limits _{\gamma _{1} =\gamma _{m}}^{\gamma _{M}} {\sum \limits _{\gamma _{2} =1}^{\gamma _{1}} {\varepsilon ^{T}(t-\gamma _{2} T)\tilde {Q}\varepsilon (t-\gamma _{2} T),} }\end{align*} And \begin{equation*} V_{5} (\varepsilon (t),t)=\sum \limits _{\gamma _{1} =1}^{\gamma _{M} } {\sum \limits _{\gamma _{2} =1}^{\gamma _{1}} {g^{T}(t-\gamma _{2} T)\tilde {R}g(t-\gamma _{2} T)}}.\end{equation*}

The remainder of proof is similar to Theorem. 3, and it is omitted.

Remark 11:

In this paper, Markov switching, cooperative-competitive relation and time delay are all considered. Different from literature [11], [13], [52], [53], [54], [55], [56], [57], our models are more related to real situations. For instance, the dynamics of agents are homogeneous in [11]. In [13], the topology structure of MASs is supposed to be fixed. In [52], the topology structure of MASs is supposed to be a special fixed switching situation. In [53], the topology structure of MASs is supposed to satisfy the condition of in- degree balance. In [54], [55], [56], and [57], the relations between agents are supposed to be cooperative or competitive. In [58], a second-order multiple agents are used to finish optimal persistent monitoring tasks. In [59], the leader-following consensus problem for a class of identical nonlinear time delay multiagent systems is studied. Paper [60] investigates the output formation-containment problem of interacted heterogeneous linear systems with leader or the follower. The discrete-time communication manner is applied to reduce the communication consumption. Based on the compensator, some algorithms are given to deal with the leader-following output consensus problem for a class of nonlinear time delay multiagent systems in [61]. Therefore, the obtained results in this paper have extended the given achievements in [11], [13], [52], [53], [54], [55], [56], [57], [58], [59], [60], and [61]. On the other hand, it is not difficult for us to find that all the sufficient conditions in the Theorem.1-Theorem.5 can be converted into solving the related LMIs. As we know, LMI is a very effective tool in dealing with some computing problems on linear matrix inequalities. Hence, the obtained results give us some very effective methods in dealing with the problem of couple-group consensus for HMASs.

SECTION IV. Some Numerical Examples

In this section, some numerical examples will be given to verify the validity of the obtained results.

Example.1:

In this example, the couple group consensus of HMASs (7) and (8) with Markov switching will be discussed. The Markov transitive rate matrix $\Lambda $ is given as follows.\begin{align*} \Lambda =\left [{ {{\begin{array}{cccccccccccccccccccc} {-\frac {2}{3}} &\quad {\frac {1}{6}} &\quad {\frac {1}{6}} \\[0.5pc] {\frac {1}{4}} &\quad {-\frac {1}{2}} &\quad {\frac {1}{4}} \\[0.5pc] {\frac {1}{4}} &\quad {\frac {1}{4}} &\quad {-\frac {1}{2}} \\ \end{array}}} }\right]\end{align*}

The Markov switching network is composed of three sub topologies as Fig. 1. In each sub topology, there are five agents. Agents 1,2 and are 2nd-order, agents 4 and 5 are 1st-order. The five agents are divided into two subgroups: G₁ = {1,2,3} and G₂ = {4,5}. In Fig. 1.(a), there is a spanning tree: $0\to 1\to 2\to 3\to 5\to 4$ . In Fig. 1.(b), there is a spanning tree: $\begin{aligned} 0\to 1\to {{\begin{cases} {2\to 3} \\ {5\to 4} \\ \end{cases}}} \end{aligned}$ . In Fig. 1.(c), there is a spanning tree: $\begin{aligned} 0\to 1\to {{\begin{cases} {3\to 2} \\ {5\to 4} \\ \end{cases}}} \end{aligned}$ .

FIGURE 1.

The topology of HMASs (7) and (8).

The adjacent matrices $A^{1},A^{2}$ and $A^{3}$ of the three sub topologies are described as follows.\begin{align*} A^{1}=&\left [{ {{\begin{array}{cccccccccccccccccccc} 0 &\quad 1 &\quad 0 &\quad 0 &\quad 1 &\quad 0 \\ 0 &\quad 0 &\quad 1 &\quad 0 &\quad 0 &\quad 0 \\ 0 &\quad 0 &\quad 0 &\quad 1 &\quad 0 &\quad 0 \\ 0 &\quad 1 &\quad {1.2} &\quad 0 &\quad 0 &\quad 1 \\ 0 &\quad 0 &\quad 0 &\quad 0 &\quad 0 &\quad 2 \\ 0 &\quad 0 &\quad 0 &\quad 0 &\quad 1 &\quad 0 \\ \end{array}}} }\right], \\ A^{2}=&\left [{ {{\begin{array}{cccccccccccccccccccc} 0 &\quad 1 &\quad 0 &\quad 0 &\quad 1 &\quad 0 \\ 0 &\quad 0 &\quad 1 &\quad 0 &\quad 0 &\quad {1.5} \\ 0 &\quad 0 &\quad 0 &\quad {1.5} &\quad 0 &\quad 0 \\ 0 &\quad {0.75} &\quad {1.5} &\quad 0 &\quad 0 &\quad 0 \\ 0 &\quad 1 &\quad {1.2} &\quad 0 &\quad 0 &\quad 1 \\ 0 &\quad 0 &\quad 0 &\quad 0 &\quad 1 &\quad 0 \\ \end{array}}} }\right], \\ A^{3}=&\left [{ {{\begin{array}{cccccccccccccccccccc} 0 &\quad 1 &\quad 0 &\quad 0 &\quad 1 &\quad 0 \\ 0 &\quad 0 &\quad 0 &\quad {1.2} &\quad 0 &\quad {1.5} \\ 0 &\quad {0.8} &\quad 0 &\quad 0 &\quad 0 &\quad 0 \\ 0 &\quad 1 &\quad {1.5} &\quad 0 &\quad 0 &\quad 0 \\ 0 &\quad 0 &\quad 0 &\quad 0 &\quad 0 &\quad 0 \\ 0 &\quad 0 &\quad 0 &\quad 0 &\quad 1 &\quad 0 \\ \end{array}}} }\right].\end{align*}

In this situation, let {$\alpha ^{\sigma (t)}$ , $\beta ^{\sigma (t)}$ } be {1,2}, {1.5,2.5} and {0.8,1.8} if $\sigma $ (t) = 1, $\sigma $ (t) = 2 and $\sigma $ (t) = 3, respectively. Let T, $\gamma _{\mathrm {M}}$ , $\gamma _{\mathrm {m}}$ , $\gamma $ , $x^{T(0)}$ and $v^{T(0)}$ be T = 0.002, $\gamma _{\mathrm {M}} =550$ , $\gamma _{\mathrm {m}} =50$ , $\gamma =100$ , $x^{T(0)} =$ {1,10,−2,−2,5}, $v^{T(0)} =$ {2,1.5,2.2}. By using Matlab LMI toolbox, a feasible solution of $P_{i}$ , Q, $S$ and $R$ have been obtained, as shown in the equations at the bottom of the page.

Hence, there exist positive and symmetric matrices $P_{\mathrm {i}} >0$ (i = 1,2,), $Q >0$ , $S >$ 0 and $R >$ 0 satisfying the linear matrix inequality (32). According to Theorem 1 and Theorem 2, the leader-following couple-group of HMASs (7) and (8) can be realized. Fig 2 has verified this kind of validity. The position and velocity trajectories of agents of HMASs (7) and (8) are shown in Fig. 2.

FIGURE 2.

The position and velocity trajectories of agents in HMASs (7) and (8) with Markov switching.

Example.2:

In this example, the Markov switching network is composed of three sub topologies as Fig. 3. In Fig. 3.(a)–​Fig. 3.(c), there is no path from 1 to 4 and there is no path from 4 to 1. Therefore, there is no spanning tree in Fig. 3.(a)–​Fig. 3.(c).

FIGURE 3.

The topologies of HMASs (7) and (8).

Similar with Example 1, the adjacent matrices $A^{1},A^{2}$ and $A^{3}$ of the three sub topologies are described as follows.\begin{align*} A^{1}=&\left [{ {{\begin{array}{cccccccccccccccccccc} 0 &\quad 0 &\quad 0 &\quad 0 &\quad 0 \\ {1.1} &\quad 0 &\quad {1.2} &\quad 0 &\quad 0 \\ 1 &\quad 0 &\quad 0 &\quad 0 &\quad 1 \\ 0 &\quad 0 &\quad 0 &\quad 0 &\quad 2 \\ 0 &\quad 0 &\quad 0 &\quad 0 &\quad 0 \\ \end{array}}} }\right],\\ A^{2}=&\left [{ {{\begin{array}{cccccccccccccccccccc} 0 &\quad 0 &\quad 1 &\quad 0 &\quad 0 \\ {0.8} &\quad 0 &\quad {1.3} &\quad 0 &\quad 0 \\ {1.2} &\quad 0 &\quad 0 &\quad 0 &\quad {1.5} \\ 0 &\quad 0 &\quad 0 &\quad 0 &\quad {1.7} \\ 0 &\quad 0 &\quad 0 &\quad 0 &\quad 0 \\ \end{array}}} }\right], \\ A^{3}=&\left [{ {{\begin{array}{cccccccccccccccccccc} 0 &\quad 0 &\quad {1.2} &\quad 0 &\quad 0 \\ {1.5} &\quad 0 &\quad {1.4} &\quad 0 &\quad 0 \\ 0 &\quad 0 &\quad 0 &\quad 0 &\quad {1.3} \\ 0 &\quad 0 &\quad 0 &\quad 0 &\quad 2 \\ 0 &\quad 0 &\quad 0 &\quad 1 &\quad 0 \\ \end{array}}} }\right].\end{align*}

Based on Theorem. 4 and pinning control strategy (59), let the coefficients of pinning control be $d_{1} =0.75,d_{2} =d_{3} =d_{5} =0,d_{4} =1.5$ , and \begin{align*}&\{\alpha ^{\sigma (1) },\beta ^{\sigma (1) }\}=\{1.1,2.1\},\\&\{\alpha ^{\sigma (2) },\beta ^{\sigma (2) }\}=\{1.6,2.6\},\\&\{\alpha ^{\sigma (2) },\beta ^{\sigma (3) }\}=\{0.9,1.9\},\quad T=0.002,\quad \gamma _{M} =600, \\&\gamma _{m} =75,\quad \gamma =100,\\&x^{T}(0)=\{1,10,-2,-2,5\}, v^{T}(0)=\{2,1.5,2.2\}, \\&x_{\bar {\omega }i} =2.1,\quad i\in \{1,2,3\}, \\&x_{\bar {\omega }i} =-2,\quad i\in \{4,5\}.\end{align*} By using Matlab LMI toolbox, a feasible solution for $\tilde {P}_{i}, \tilde {Q},\tilde {S}$ and $\tilde {R}$ has been obtained, as shown in the equations at the bottom of the next page.

Fig. 4 has described the position and velocity trajectories of agents in HMASs (7) and (8) under pinning control (59). And Fig. 4.also shows the validity of Theorem. 4.

FIGURE 4.

The position and velocity trajectories of agents in hmass (7) and (8) under pinning control (58).

Example 3:

In this example, the topology of HMASs (7) and (8) is composed of three parts as Fig. 5. Obviously, there is no any spanning tree in Fig. 5(a)–​Fig. 5(c). In Fig. 5(a), the out degrees of node 1and node 5 are zero. Both the in degrees of node 1 and node 5 are two. This is to say, Fig. 5(a) cannot satisfy the balance of degree. And in pinning control protocol (59), neither of node 1 and node 5 is pinned. Other parameters are completely set as Example 2. Fig. 6 shows that the couple group consensus of HMASs (7) and (8) cannot be achieved if the node 1 and node 5 are not pinned.

FIGURE 5.

The topological structure of HMASs (7) and (8).

FIGURE 6.

The position and velocity trajectories of agents in HMASs (7) and (8).

Remark 12:

In Example.1, there exists a spanning tree in each subgraph, but each sub-topology cannot satisfy the balance of degree and they are not also strong connectivity. In Example.2, there is no any spanning tree in each subgraph. Compared with the obtained results, the demand of topology can be relaxed in this paper. Example.3 shows that the nodes with zero degree must be pinned. Otherwise, the couple-group of HMASs (7) and (8) will not be realized.

SECTION V. Conclusion

Based on signed digraphs, paper [62] has studied the tracking control problem for second-order MASs with communication constraint scenarios in real engineering applications on tracking dynamics, including external disturbances, asynchronous clocks, and switching interactive topologies. In [63], a novel distributed control scheme is designed for the Cucker-Smale model to examine the leader-follower flocking behavior in cooperative-competitive networks. In [64], event-triggered control is used to investigate the synchronization for a class of switched uncertain neural networks (NNs) with frequent asynchronism. In [65], leader-following consensus for a class of high-order nonlinear multi-agent systems has been studied. Some non-lipschitz continuous control laws have been proposed to realize the finite-time consensus of high-order uncertain nonlinear multi-agent systems. Compared with [58], [59], [60], [61], [62], [63], [64], and [65], leader-following and pinning control couple-group consensus of a kind of HMASs with Markov switching have been studied in this paper, especially cooperative-competitive relation, heterogeneous and Markov switching are taken into account. Based on cooperative and competitive relation and Markov switching, some novel couple group consensus protocols have been designed for this HMASs. By using graph algebra theory, matrix theory and stability theory, the couple group consensus problem of HMASs with Markov switching can be converted into analyzing the stability of a new delta system. Stochastic delta operator, Lyapunov-krasovskii function and LMI (Linear Matrix Inequality) skills have been used to obtain a few sufficient conditions for couple group consensus of these HMASs with Markov switching. In the end, some examples have been presented to address the validity of our achievements. In our future work, event-triggered consensus of HMASs with Markov switching will be considered.