Distributed Optimization for Aggregative Games Based on Euler-Lagrange Systems With Large Delay Constraints

This article investigates an aggregative game based on Euler-Lagrange systems subject to time-varying communication delays. First, a distributed algorithm is put forward to try to find the Nash equilibrium by the deliberated group of Euler-Lagrange systems with “small” delay and “large” delay. Second, we illustrate the convergence of two circumstances, separately. The first circumstance derives the upper bound of delays for guaranteeing globally exponential convergence, and the other obtains globally exponential convergence, even in some restrictions on “large” delays. Finally, a numerical example is used to show the effectiveness and superiority of proposed method.


I. INTRODUCTION
Aggregative games have been enormously received a great concern in many areas owing to their wide applications in areas of smart grid [1]- [3], plug-in electric vehicles [4]- [6], congestion control in communication networks with shared resources [7], [8]. The main characteristics of the aggregative games is that the cost of each player depends on not only its own decision, but also the average among the strategies of all other players. In response to such problems as aggregative games, extensive research has been exceedingly conducted on seeking Nash equilibrium to minimize their cost function (see refs. [9]- [12] and references cited therein).
Considering the aggregation games, the method of distributed design plays an increasingly significant role in the algorithm implementation, where players (agents) exchange data through local information and between neighbors via an undirected and connected communication graph. In the aggregative games, Liang et al. [13] proposed a distributed algorithm in the presence of coupled constraints by making use of projected dynamics and non-smooth tracking dynamics. Ye and Hu [9] developed a leader-following consensus protocol, which can utilize the scheme of gradient play for The associate editor coordinating the review of this manuscript and approving it for publication was Guangdeng Zong . the noncooperative games over networks. Zhang et al. [14] considered a distributed algorithm, which can take advantage of the method of the gradient for aggregative game with uncertain perturbed nonlinear dynamics. In fact, communication or information exchange among players play a key role in solving aggregative games. Hence, the data transmission by means of wireless communication channels may suffer from the effects of time-varying delay or packet dropouts, which inevitably reduce the control performance and even instability [15], [16]. In view of this, it is of great importance to investigate time delays in aggregative games for distributed optimization.
In practice, Euler-Lagrange systems can show the motion behavior of a large class of physical systems, such as electrical, robotic, autonomous vehicles and aerospace systems. As well known, stochastic noises and structure switches widely exist in Euler-Lagrange systems [17]- [19]. Motivated by this fact, some distributed algorithms with Euler-Lagrange systems were developed. Zhang et al. [20] proposed the distributed optimal coordination algorithm, which can make the multiple heterogeneous Euler-Lagrangian systems global convergence subject to uncertainties. Sun et al. [21] developed a distributed optimization problem for a set of robots, which considered the effect of the time-varying condition and uncertain Euler-Lagrange dynamics. However, to the best of our knowledge, few results have been reported studying time delays, especially ''large'' delays about the problem of the distributed Nash equilibrium trying to find with Euler-Lagrangian systems.
Summarizing the above discussions, we aim to investigate a distributed algorithm to try to find the Nash equilibrium by the deliberated group of Euler-Lagrange systems with ''small'' delay and ''large'' delay. The main contributions of this article can be concluded as follows: Firstly, we provide an aggregative game of Euler-Lagrange systems, which are subject to transmission delays. Secondly, we illustrate a distributed algorithm for the two circumstances and their convergence are analyzed. The first circumstance derives the upper bound of delays for guaranteeing globally exponential convergence, and the other obtains globally exponential convergence, even in some restrictions on ''large'' delays.
The remainder of this article is organized as follows. In Section 2, some preliminaries and the considered problem are presented. In Section 3, we propose a distributed algorithm design under time-varying communication delays, where their convergence can be analyzed. Furthermore, a numerical example is used to demonstrate the effectiveness of the proposed method in Section 4. Finally, we conclude this article in Section 5.
Notations : N indicates the set of natural numbers, R n stands for the n-dimensional Euclidean space and R n×m represents the set of n × m real matrices, I n indicates the n × n identity matrix. C ς ⊗ D ς stands for Kronecker product of matrices C ς and D ς . · represent the Euclidean norm for vectors in R n . For a matrix M ς , M T ς shows the transpose. A matrix M ς > 0 ( M ς ≥ 0 ) indicates that matrix M ς is a symmetric positive definite (semi-positive definite) matrix. λ min (M ς ) (λ max (M ς )) stands for the minimum (maximum) eigenvalue of M ς . We use 0 denotes appropriate dimension matrix or vector, 1 n := (1, · · · , 1) T ∈ R n , 0 n := (0, · · · , 0) T ∈ R n , and denote col(x 1 , · · · , x n ) as the column vector.

II. PRELIMINARIES AND PROBLEM STATEMENT A. PRELIMINARIES
In this section, we can be briefly introduce some basic information relied on graph theory [22] and convex functions [23]. In addition, some useful definitions on ''small'' and ''large'' delays are also presented.
We assume a weighted graph G ς = (V ς , E ς , A ς ), which can be composed of a finite node set V ς = {1, 2, · · · , N }, the edge set E ς ∈ V ς × V ς , and the adjacency matrix A ς . We consider an edge from k σ to l σ , expressed by (k σ , l σ ), indicates that k σ can obtain the transmitted data from l σ , and l σ is regarded as a neighbor of k σ . We define A ς = [a ς kσ lσ ] ∈ R N ×N as a weighted adjacency matrix of G ς with a k σ k σ = 0(no self-loop), When (k σ , l σ ) ∈ E ς , then a ς kσ lσ > 0, and otherwise a ς kσ lσ = 0. When a ς kσ lσ = a ς lσ kσ , then the graph is viewed as undirected. We define the degree matrix a k σ l σ for k σ = 1, · · · , N , and the Laplacian matrix of G ς can be expressed as L ς = D ς − A ς . Obviously, L ς 1 N = 1 T N L ς = 0. The eigenvalues of L ς are denoted by λ ς 1 , . . . λ ς N . For a connected undirected graph, no less than one of the eigenvalues of the Laplacian matrix L ς is zero and the eigenvalues without zero are positive with the corresponding eigenvector space Suppose that a function f (·) : R m → R, we can say that f (·) is convex if the following inequality holds: Suppose that a function f (·) : R m → R m , f (·) is said to be β ζ − strongly monotone (β ζ > 0) on R m if the following inequality holds: Next, we introduce the definitions of ''small'' delay and ''large'' delay as follows: The updating sequence {t ς σ } can be represented as {t 0 , t 1 , · · · }, where 0 ≤ t 0 < t 1 < · · · . Then, based on this updating sequence, we define two sets as follows: where E sdp stands for the set of ''small'' delay period, E ldp represents the set of ''large'' delay period.

B. PROBLEM STATEMENT
In this work, we investigate an aggregative game with a set of N players (or agents) indexed by 1 · · · N . All players can be viewed as the node set V ς = {1, . . . , N }, where information exchange between them is described via an undirected connected graph G ς . Then, the following two factors can be introduced mainly.
, which depends on not only player k σ decision δ ς k σ (t) ∈ R m , but also the average among the strategies of all other players. 2) Strategy: Each player aims at minimizing its local cost function ) by seeking an appropriate strategy δ ς k σ (t). We define the strategy profile of this game is δ ς (t) col δ ς 1 (t), . . . , δ ς N (t) and aggregate In this article, our objective is to minimize the cost function For the aggregative game, the definition introduced below is based on the Nash equilibrium [2].
Definition 1: For the aggregative game, a strategy profile δ ς

can be regraded as a Nash equilibrium if
The definition 1 tells us that all players simultaneously are to minimize their cost function at δ ς * (t), where via changing its individual decision unilaterally, no player can further obtain their own feasible benefits.
Moreover, for an aggregative game (1) to find the Nash equilibrium seeking problem, we define as is not only continuously differentiable in δ ς (t) and but also convex in is β ς −strongly monotone, and then there exists a constant κ, such that κ > 0 on R N (m+n) .
Remark 1: The above Assumption 3 and Assumption 4 are used to deal with the distributed Nash equilibrium seeking problem (see [2], [12], [13], [24]). On the premise that Assumption 3 and Assumption 4 can be established, we can next obtain Lemma 1 and Lemma 2.
Lemma 2 [12]: Under the premise that Assumption 3 is established, considering the problem of the aggregative game (1), δ ς Remark 2: As far as the problem of the aggregative game is concerned, Lemma 1 illustrates that the solution of the Nash equilibrium is existence as well as uniqueness. The condition given by Lemma 2 is essential to ensure the existence of the Nash equilibrium point.
In addition, we can introduce the dynamics of player k σ (k σ ∈ V ς ) by means of the Euler-Lagrange equation as follows: where δ ς k σ (t) ∈ R m is the generalized coordinates vector. T ς k σ (δ ς k σ (t)) ∈ R m×m is the symmetric positive definite inertia matrix; C ς k σ (δ ς k σ (t),δ ς k σ (t))δ ς k σ (t) ∈ R m represents the Coriolis and centripetal force vector; g ς k σ (δ ς k σ (t)) ∈ R m denotes the gravitational force; and τ ς k σ (t) ∈ R m is the control input on the k σ th player. As is well known, the Euler-Lagrange systems have the following characteristics [25]: 1) Skew symmetry property: R p is composed of constant parameters vector for unknown associated with the player k σ . In multi-agent systems, communication or information exchange among players plays a decisive role in solving optimization problems. In order to look for the Nash equilibrium, each agent of the aggregative game (1) cooperatively and selectively shares information with other agents by means of a network G ς = (V ς , E ς , A ς ) where time delaysκ σ (t) are inevitable in communication networks. Owing to wireless communications suffered from the effects of time-varying delay, particularly ''large'' delays, it may result in undesirable dynamics and even instability. Thus, taking into account the aggregative game (1), it is significant that a distributed algorithm is put forward to try to find the Nash equilibrium subject to communication delaysκ σ (t).

III. MAIN RESULTS AND PROOFS
Now, considering the multi-agent system (5), our objective of this section is to put forward a distributed algorithm to try to find the Nash equilibrium with respect to ''small'' delay and ''large'' delay. Then, their convergence are analyzed.

A. EULER-LAGRANGE PLAYERS WITH ''small'' DELAY
The distributed Nash equilibrium seeking algorithm subject to wireless communications accompanied by time-varying delays for player k σ (k σ ∈ V) is proposed as follows: wherek µ > ×W k σ (δ ς k σ (t), ς k σ (t)) is the game. ς k σ (t) is the estimation of µ(δ ς (t)), and v ς ς k σ (t), z ς k σ (t) are auxiliary variables for estimation, which play the role of sharing local data with neighboring players.κ σ (t) is network-induced time-varying delay, which is viewed as piecewise continuous.
As far as the algorithm (6) is concerned, for the sake of simplicity, we consider that the network-induced timevarying delays caused by neighbors of player k σ (k σ ∈ V ς ) are the same.
We defineq . Then, by combining (5) and (6), the multi-agent system (5) can be reformulated in a compact form as follows: where
Remark 3: Lemma 3 is essentially necessary to assure that when (8) converges to its equilibrium points, based on the multi-agent system (5), the problem of Nash equilibrium associated with the algorithm (6) is close to the Nash equilibrium modelled by the aggregative game (1).
Proof: Before starting this proof, we first consider coordinate transformation.

B. EULER-LAGRANGE PLAYERS WITH ''large'' DELAY
In this work, considering the multi-agent system (5), in order to try to find Nash equilibrium, a distributed algorithm is introduced subject to wireless communications accompanied by large delay, namely h <κ σ (t) ≤κ σ , and give convergence analysis for our proposed algorithm. This algorithm allows ''small'' induced delay and ''large'' time delay selectivity to occur within a given period of time, which assumes ''small'' induced delay can take place all the time, whereas the frequency of ''large'' induced delay is confined. Considering the presence of ''large'' induced delay, the distributed Nash equilibrium seeking algorithm subject to wireless communications accompanied by time-varying delays for player k σ (k σ ∈ V ς ) can be formulated by the switched delay system as follows: where γ σ (t) is a switching signal, such that γ σ (t) ∈ [0, +∞) and when t ∈ E sdp , γ σ (t) = 1; when t ∈ E ldp , γ σ (t) = 2, κ σ k σ (t), k σ ∈ {1, 2} are the networked induced delay and satisfy Assumption 1. Then, we define the following the subsequences of where l k σ , l p σ are positive constants (k σ ≤ p σ < i σ ). Assumption 5: We limit the scope of the two concepts, namely the ''large'' induced delay length rate and its occurrence frequency.
Proof: We consider the same coordinate transformation as (8). Then, we can obtaiṅ whereξ ς (t), A ς , B ς and ϕ(θ ς ) are the same as that in (13) and (15). Then, based on the algorithm (23), we choose the following the Lyapunov function to prove the multi-agent system (5) ultimately exponentially converges with large induced delay.
Since system (23) is a switching time-delay system, that is, there exists a switching signal (24), we discuss them separately.
Then, derivative ofṼ ς d (t) getṡ Becausẽ (28) Then, we can deduce thaṫ Secondly, when the switching signa γ σ (t) = 2, we can calculate thaṫ Then, we can geṫ From (27) and (32), we can conclude that On the premise that we combine (29), (32) and (34), for given a constant * > 1 satisfying the following inequality: This means that the result of ultimately exponential convergence can be ensured.

IV. SIMULATION RESULTS
In this work, a numerical example is supplied to verify the effectiveness of the proposed theoretical scheme.
The electric vehicle market has been growing rapidly around the world. An aggregative game has been proposed for satisfying the electric vehicle charging demand in the electricity market [26], [27]. Based on the fact that wireless communications are vulnerable to time-varying delay, we present an example of the aggregative game, which exists six generation systems and illustrate the communication topology in Fig. 1.
The cost function of the generation system k σ (k σ ∈ V ς ) shows where P ς k σ ∈ R represents the output power of the k σ th generation system, P ς −k σ col P ς 1 , . . . , P ς k σ −1 , P ς k σ +1 , . . . , P ς N , c k σ (P ς k σ ) and p(µ) denote the generation cost and the electricity price, respectively. Borrowed from [28], we can obtain  where ν 1 , ν 2 and ν 3 are the cost coefficients of the generation system k σ . On the other hand, there exists two constants p 0 , g, In [29], in terms of the ith turbine-generator system, the dynamics model can be presented as follows: where K mk σ denotes the gain of the machine's turbine k σ , T mk σ represents the time constant of the machine's turbine k σ , in s, X ek σ stands for the opening degree of the steam valve of generator k σ , in p.u. ω k σ shows the relative speed of the generator k σ , in rad/s, ω 0 indicates the synchronous machine speed, in rad/s, K ek σ indicates the gain of the speed governor k σ of the machine, T ek σ means the time constant of the speed governor k σ of the machine, in s, R k σ denotes the regulation constant of the machine k σ , in p.u, P ck σ means the power control input of the generator k σ , in p.u. D k σ denotes the damping constant per unit, H k σ means the inertia constant, in s. Table 1 presents the parameters of the generator k σ (k σ ∈ V ς ). The system initialization parameters are as follows: ω 0 = 314.159, p 0 = 200, g = 0.1, δ ς k σ (0), ς k σ (0), v ς ς k σ (0), z ς k σ (0) are all zeros. Choosek µ = 8, γ ς = 1 such that the inequality (14) holds. Choose l e = 0.6, l d = 1.01, then, we have h 0.01.
Accordingly, it is noticed that Figs. 2-4 demonstrate the simulation results. As shown in Fig. 2, the output power of   the generation system k σ (k σ ∈ V ς ) ultimately converges to the Nash equilibrium modelled by the aggregative game under ''small'' delay. Fig. 3 illustrates that when ''large'' delays occur frequently, the output power of the generation system k σ (k σ ∈ V ς ) cannot ultimately converge to the Nash equilibrium modelled by the aggregative game. However, on the premise that the switching signal γ σ (t) can be established, from the Fig. 4, it is concluded that the output power of the generation system k σ (k σ ∈ V ς ) can ultimately exponentially converge to the Nash equilibrium modelled by the aggregative game even though the presence of ''large'' delay.

V. CONCLUSION
This article investigates the issue of an aggregative game based on Euler-Lagrange systems subject to time-varying communication delays. To solve the problem, firstly, a distributed algorithm is put forward to try to find the Nash VOLUME 8, 2020 equilibrium by the deliberated group of Euler-Lagrange systems with ''small'' delay and ''large'' delay. Secondly, we illustrate the convergence of two circumstances, separately. The first circumstance derives the upper bound of delays for guaranteeing globally exponential convergence, and the other obtains globally exponential convergence, even in some restrictions on ''large'' delays. Finally, a numerical example is used to show the effectiveness and superiority of proposed method.