An Incremental Input-to-State Stability Condition for a Class of Recurrent Neural Networks

This article proposes a novel sufficient condition for the incremental input-to-state stability of a class of recurrent neural networks (RNNs). The established condition is compared with others available in the literature, showing to be less conservative. Moreover, it can be applied for the design of incremental input-to-state stable RNN-based control systems, resulting in a linear matrix inequality constraint for some specific RNN architectures. The formulation of nonlinear observers for the considered system class, as well as the design of control schemes with explicit integral action, are also investigated. The theoretical results are validated through simulation on a referenced nonlinear system.

Abstract-This article proposes a novel sufficient condition for the incremental input-to-state stability of a class of recurrent neural networks (RNNs).The established condition is compared with others available in the literature, showing to be less conservative.Moreover, it can be applied for the design of incremental input-to-state stable RNN-based control systems, resulting in a linear matrix inequality constraint for some specific RNN architectures.The formulation of nonlinear observers for the considered system class, as well as the design of control schemes with explicit integral action, are also investigated.The theoretical results are validated through simulation on a referenced nonlinear system.Index Terms-Linear matrix inequalities, neural networks (NNs), nonlinear control systems, stability of nonlinear systems.

I. INTRODUCTION
N EURAL networks (NNs) have gained interest in many engineering fields, given the ever-growing availability of large amounts of data, e.g., collected measurements from plants, and their significant ability to reproduce nonlinear dynamics [1], [2].In particular, NNs have shown to be particularly suited for control applications [3], [4], [5], [6], as they can be used not only to identify unknown systems, but also to directly design feedback controllers from data [7], [8].Among existing NN architectures, recurrent neural networks (RNNs) are typically adopted for controlling dynamical systems, since they are inherently characterized by the presence of state variables [9].
Despite the increasing popularity of RNNs, their theoretical properties have been rarely analyzed.As nonlinear dynamical systems, it is in fact fundamental to characterize conditions that guarantee the stability of their motions, especially when RNNs are part of control systems [10], [11].In this context incremental input-to-state stability (δISS) [12], [13] plays a crucial role.This property entails that, asymptotically, the state trajectories are solely determined by the applied inputs and not by their initial conditions [14].Thus, the dynamics of a δISS RNN is asymptotically independent of its initialization.The δISS property also enables the design of trivial observers for the RNN states.The latter, indeed, can be asymptotically estimated just exploiting the knowledge of the applied inputs.Finally, note that the δISS implies other common stability properties, e.g., global asymptotic stability (GAS) of the equilibria and input-to-state stability (ISS) [12], [14].
Motivated by this, the article presents a novel δISS sufficient condition for a class of RNN architectures.The proposed condition is applicable to control systems, and in particular for the analysis and design of RNNs-based feedback controllers and feedforward compensators.

A. State of the Art and Contribution
Despite the large popularity and potentialities of RNNs in control applications, relatively few stability results have been discussed in the literature.Sufficient conditions ensuring stabilityrelated properties for RNNs were presented in [15] and in [16], the latter focusing on a specific class of RNNs, i.e., gated recurrent units.A stability condition for a class of RNNs was discussed in [17], considering however the case with constant inputs.Note that the abovementioned contributions address stability properties weaker than the δISS (e.g., the GAS property), which do not consider the effect of inputs [12].This has motivated other research studies to focus on conditions guaranteeing δISS.The latter, interestingly, can be directly enforced in the data-based RNN training phase, e.g., as discussed in [18] and [19].Also, sufficient conditions guaranteeing alternative contraction properties were provided in [20] for echo state networks (ESNs) and in [21] and [22] for more general RNNs.However, these works focus on open-loop RNNs, and they do not address the design of stabilizing RNN-based feedback controllers.
Regarding control systems, in [23] the stability was analyzed in case of FeedForward NN (FFNN) controllers and assuming a linear controlled system with uncertainties.Design conditions for FFNN controllers were also provided in [24], considering specific classes of second-order nonlinear systems under control.Also, model predictive control (MPC) has been investigated as a method for the design of efficient controllers applicable to systems described by specific classes of RNNs.For instance, the ISS of a MPC-controlled neural nonlinear autoregressive exogenous (NNARX) system was discussed in [25].Also, MPC regulation strategies for other RNN architectures were presented in [26] and [27], ensuring closed-loop stability if the RNN-based model of the controlled system enjoys the δISS property.
In this work we first derive a novel δISS condition for a class of discrete-time nonlinear systems, which includes the one analyzed in [28], as well as different common RNN classes, e.g., ESNs and NNARXs.We prove that the proposed condition is less conservative than existing ones established in the past years for RNNs lying in the considered class (or in slightly more general classes), e.g., [17], [18], [20], [21].The established results turn out to be particularly suited for the design of feedback controllers and feedforward compensators.In particular, they allow us to enforce the δISS property on control systems, also in case the controlled system does not enjoy the same property.Moreover, we show that, if specific RNN-based control architectures are considered, the design problem translates to a linear matrix inequality (LMI) problem, efficiently solvable by common solvers.

B. Article Outline
The rest of this article is organized as follows.In Section II, the equations of the considered class of RNNs are introduced.The novel δISS sufficient condition is stated in Section III and it is compared with other existing conditions in the literature.In Section IV, the δISS properties of feedforward and feedback interconnected RNNs are investigated, whereas Section V discusses in details the controller design with δISS guarantees.Section VI shows the application of the theoretical results in this article to a simulation example.Finally, Section VII concludes this article.

C. Notation and Basic Definitions
Given a matrix A, its transpose is A T , the transpose of its inverse is A −T , its induced 2-norm is A , and its largest singular value is σ(A).The entry in the ith row and jth column of a matrix A is denoted as a ij .|A| denotes a matrix whose entries are |a ij |, for all i, j, where |a ij | is the absolute value of a ij .The ith entry of a vector v is indicated as v i .Given a symmetric matrix P , we use P 0, P 0, P 0, and P ≺ 0 to indicate that it is positive semidefinite, positive definite, negative semidefinite, and negative definite, respectively.λ min (P ) and λ max (P ) denote the minimum and maximum eigenvalues of a symmetric matrix P , respectively.0 n,m denotes a zero matrix with n rows and m columns and I n is the identity matrix of dimension n.Given a sequence of square matrices . ., A n as submatrices on the main-diagonal blocks.D denotes the set of diagonal matrices whose dimension is clear from the context.Also, v = √ v T v denotes the 2-norm of a column vector v and v Q = v T Qv denotes the weighted Euclidean norm of v, where Q is a positive definite matrix.Given a sequence u = u(0), u(1), . . ., we define its infinity norm as u ∞ = sup k∈N u(k) .Also, id n (•) denotes a column vector of dimension n with all elements equal to the identity function id(•).We introduce the following definition.
Definition 1 (see [29]): A real function g : R → R is called globally Lipschitz continuous if there exists a constant L p ≥ 0 such that, for any x, y ∈ R, it holds that The following property will be used later in the article.
for any scalar τ = 0. We now consider a general discrete-time nonlinear system expressed as where x ∈ X is the state of the system, and u ∈ U is the exogenous variable.The set of admissible input sequences u is denoted by U .We indicate with x(k, x 0 , u) the solution to the system (1) at time k starting from the initial state x 0 ∈ X with input sequence u ∈ U. Now, we recall some useful notions for the following (see [14]).

II. PROBLEM STATEMENT
We consider the following class of nonlinear discrete-time systems: where u ∈ R m is the exogenous variable, y ∈ R l is the output vector, x ∈ R n is the state vector, Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
R n is a vector of scalar functions applied element-wise, Given a system in class (4), let us introduce the set Note that, under Assumption 1, system (4) is representative of several RNN architectures.For instance, (4) includes the general formulation of RNNs considered in [28], where  [30] are particular types of RNNs composed of a dynamical reservoir (hidden layer) in which the connections between neurons are sparse and random.If we consider the formulation proposed in [26], with ν neurons χ ∈ R ν , input u ∈ R m , output y ∈ R l , Lipschitz continuous internal units output functions ζ(•) applied element-wise, and linear output units output functions, the ESN equations are where Note that model (6) can be reformulated as (4) by defining

, and by setting
Shallow NNARX models: NNARX is a class of nonlinear autoregressive exogenous models where a FFNN is used as nonlinear regression function.As shown in [18], a shallow (i.e., single-layer) NNARX, with input ũ ∈ R m , output y ∈ R l , and ν neurons, is a dynamical system defined by the following equation: where Note that model (8) can be reformulated as (4).To do that, let us introduce the following vectors: Consequently, by setting and it is evident that model (8) belongs to the class (4).Example 3. Class of RNN systems in [17]: In [17], a slightly different RNN class was considered, i.e., where A is a full matrix, E = diag(e 1 , . . ., e n ), s = [s 1 . . .s n ] T is a vector of constant inputs, with e i , s i ∈ R, and where each f i (•) is a globally Lipschitz continuous and monotone nondecreasing activation function with Lipschitz constant L pi .Although (12) is slightly more general, to perform a comparison, we write system (12) in class (4a) by setting E = 0 n,n .It is worth noting that the latter special case of system (12) matches the RNN formulation addressed in [28] or the ESN formulation in [20], in case of constant inputs.
Example 4. Class of contracting implicit RNNs (ci-RNNs) in [21]: In [21], a slightly more general RNN class was taken into account.In particular, in the single layer case, the following contracting implicit RNN is considered Also in this case, to perform a comparison, we write system (13) in class (4) by setting , where Φ(•) contains a nonlinear activation with Lipschitz constant L p = 1 (for simplicity, cf.[21, Section 2]).
Given the class of systems (4) under Assumption 1, a sufficient condition ensuring the δISS property is established and described in the following section.

III. A NOVEL SUFFICIENT CONDITION FOR INCREMENTAL INPUT-TO-STATE STABILITY OF RNNS
Let us consider a generic system (4) fulfilling Assumption 1.The following theorem provides a sufficient condition which guarantees the δISS for nonlinear systems lying in the class (4a).We first define a diagonal matrix W := diag(L p1 , . . ., L pn ) ∈ R n×n , where L pi = 1 for all i / ∈ W. We introduce the matrices

. , n} with j = i, and
Proof: In order to prove the δISS of system (4a) we show the existence of a dissipation-form δISS Lyapunov function.We consider, as candidate, P .From now on, for notational simplicity, we drop the dependence on k.If we consider the is easily verified.To prove that V (x 1 , x 2 ) satisfies also condition (3), we introduce the following notation: where 15), we obtain that Now, we can observe that where q, r ∈ R n , with The elements of the vectors q and r are Therefore, by setting p ij = p ji = 0 ∀i ∈ W and ∀j ∈ {1, . . ., n} with j = i, we can compute from ( 17) that Inequality ( 18) holds since, in view of Assumption 1, f i (•) are globally Lipschitz continuous functions, for all i ∈ W, and p ii > 0 ∀i since P = P T 0.
As a result, by exploiting (18) and in view of Property 1, for any τ = 0, we can write that for any λ u > λ max (B * ), where A * := P −(1+τ 2 ) A T P A 0 by selecting a τ such that 0 < τ 2 < λ min (P − A T P A) λ max ( A T P A) , with λ min (P − A T P A) > 0 in view of ( 14), and In short, Theorem 2 ensures that system (4a) is δISS if A is Schur stable and if there exists a matrix P with a specific structure fulfilling the Lyapunov inequality (14).In particular, P must have zero elements along all the rows and columns (except for the diagonal element) corresponding to the rows of (4a) whose activation function is nonlinear.
The δISS condition in Theorem 2 is now compared with other existing conditions for the RNN systems introduced in Section II.
Proof: See the Appendix.Note that the condition in Theorem 2 applies to a more general class of models since we can have W = {1, . . ., ν} and L pi = 1 for some i ∈ {1, . . ., ν}. [18] for Shallow NNARX Models

B. Comparison With the Stability Condition in
In case of shallow NNARXs with nonlinear activation functions, the assumptions of Theorem 2 require the existence of a symmetric positive definite matrix P fulfilling (14) such that p ij = p ji = 0, ∀i ∈ W = {n − ν + 1, . . ., n} and ∀j ∈ {1, . . ., n} with j = i.
In [18,Th. 8] a sufficient condition for δISS of a deep (i.e., M -layered) NNARX is proposed.For comparison purposes, we recall here the condition for a single-layer NNARX (8), as this belongs to the class (4), where ζ(•) has activation functions with Lipschitz constant L p .Note also that, in [18], a slightly different state-space formulation with respect to the one in this article is considered.
Below we show that the assumptions of Theorem 2 are less conservative than the one of Proposition 3 for a shallow NNARX.
Proposition 4: Let ζ(•) in system (8) be a vector of nonlinear Lipschitz continuous functions with Lipschitz constant L p .If the condition of Proposition 3 holds, then there exists a matrix P fulfilling the assumptions of Theorem 2.
Proof: See the Appendix.
Here, we show an example in which the assumption of Proposition 3 is not fulfilled whereas the assumptions of Theorem 2 hold.Let we have that the maximum eigenvalue of A T P A − P is equal to −0.239 and, thus, A T P A − P ≺ 0.

C. Comparison With the Stability Condition in [17] for a Class of RNN Systems
The work in [17] analyzes the stability properties of a slightly different RNN class represented by (12).In [17, Corollary 1], some sufficient conditions for global exponential stability were proposed.Among them, the following condition has a similar structure to the one of Theorem 2, and it is thus compared.
For the sake of comparison, we now show that the assumptions of Theorem 2 are less conservative than the one of Proposition 5 for the simplified class of RNN systems ( 12) with E = 0 n,n .Therefore, the following proposition is stated.
Proposition 6: Let E = 0 n,n in system (12).If the condition of Proposition 5 holds, then there exists a matrix P fulfilling the assumptions of Theorem 2.
Proof: See the Appendix.
Here, we show an example in which the assumption of Proposition 5 is not fulfilled whereas the ones of Theorem 2 hold.
Hence, the spectral radius of M is equal to 1.225 and so ( 19) is never fulfilled.With the choice we have that the maximum eigenvalue of A T P A − P is equal to −0.1135 and, thus, A T P A − P ≺ 0.

D. Comparison With the Contraction Condition in [21] for Shallow ci-RNNs
In [21, Eq. ( 10)] the following sufficient condition for the contraction property (i.e., guaranteeing that, for any input signal, initial conditions are forgotten exponentially) was proposed.
Proposition 7 (see [21]): System (13), with E = I n , is contracting if there exists a diagonal matrix P = P T 0 such that Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
The following result shows that the assumptions of Theorem 2 are less conservative than the condition in Proposition 7.
Proposition 8: If the condition of Proposition 7 holds for system (13), with E = I n and Lipschitz constant L p = 1, then there exists a matrix P fulfilling the assumptions of Theorem 2.
Proof: See the Appendix.
Here, we show an example in which the assumption of Proposition 7 is not fulfilled whereas the ones of Theorem 2 hold.Let n = 2 and W = I 2 .By choosing

IV. δISS OF INTERCONNECTED RNNS
In this section, we consider the case where more systems in the class defined by ( 4) are connected through different interconnection schemes (i.e., in series and in feedback).As a result, we will show that the overall model of the composite system lies itself in the class defined by ( 4), implying that its stability properties can be established by resorting to Theorem 2. This has the important implication that a system in the class defined by ( 4) can be controlled through suitable feedback controllers and feedforward compensators in class (4), and one can use Theorem 2 to establish stability properties of the control scheme.This will pave the way to the control design conditions discussed in Section V.

A. Feedforward Interconnection
In this section, we investigate the δISS conditions of the series of M s systems lying in the class (4).Specifically, each system is numbered in increasing order with respect to i and is defined by where We can state the following result.Proposition 9: The series of M s systems in the class (4) lies in the class (4).
Proof: First, note that the input of the first subsystem is the input of the overall series interconnection, i.e., u(k) = u 1 (k), whereas the output of the last subsystem of the series is the overall output, i.e., y(k) = y M s (k).Since y i (k) = u i−1 (k), for all i = 2, . .., M s , due to the series interconnection, the second subsystem can be written as 1 YALMIP [32] and MOSEK [33].
Fig. 1.Feedback control scheme: C is the controller, S is the system to be controlled, r is the reference signal, e is the tracking error, u s is the manipulated variable, and y s is the output of the system.
Following the same reasoning, for i = 3, . .., M s , it holds that From ( 21), (22), and (23), by introducing the extended state vector x(k) = [x 1 (k) T . . .x M s (k) T ] T , it is possible to construct the matrices A, B, C, D and the vector f T of the overall system in the form (4).This concludes the proof.
To guarantee the δISS of the series of M s systems in the form (4), one way is to write the overall series system as (4), and then to impose the sufficient condition for δISS in Theorem 2 to the overall system.Alternatively, we can impose the sufficient condition for δISS in Theorem 2 to each subsystem.In [12, Proposition 4.7], a theoretical result proved that the series interconnection of two δISS continuous-time systems is δISS.The same property can be extended to discretetime systems.In the following Proposition 10 we show that, given a series of discrete-time systems, each one satisfying the assumptions of Theorem 2 for δISS, their series interconnection satisfies the same assumptions.
Proposition 10: Let us consider a series of systems in the class (4), each one fulfilling the assumptions of Theorem 2. The series of these systems satisfies the assumptions of Theorem 2.
Proof: See the Appendix.

B. Feedback Interconnection
In this section, we investigate the δISS conditions of the feedback of two systems lying in the class (4).The baseline feedback control scheme is depicted in Fig. 1, where r is the reference signal, and e is the tracking error.We define the Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
equations of the controller C as where u c ∈ R m c , y c ∈ R l c , and x c ∈ R n c .We consider the case in which the system is strictly proper to avoid algebraic loops.Thus, we define the equations of the (possibly identified) controlled system S as where u s ∈ R m s , y s ∈ R l s , and x s ∈ R n s .Let also m s = l c , and m c = l s .We can state the following result.
Proposition 11: The feedback interconnection in Fig. 1 of the systems ( 24) and (25) in the class (4) lies in the class (4).
Proof: First, note from Fig. 1 that r is overall input to the feedback interconnection, i.e., u(k) = r(k), whereas y s is the overall output, i.e., y(k we can write the equations of the overall closed-loop system in the formulation (4) through the following definitions: It is possible to show that the previous result holds also for the case in which the controller is strictly proper and the system is not (i.e., y c (k) = C c x c (k) and y s (k) = C s x s (k) + D s u s (k)) by defining In view of Proposition 11 and Theorem 2, it is possible to analyze or enforce (through the tuning of the parameters of C) the δISS property to the feedback control scheme in Fig. 1, where both the controller and the system are in the class (4).

V. CONTROLLER DESIGN WITH δISS GUARANTEES
This section discusses the design of controllers that confer δISS guarantees to the control system.In this article, we will not focus on the performances of the control system, which will be a matter of future research [34].
In general, the δISS condition ( 14) in Theorem 2 corresponds to a nonlinear constraint in control design, due to the product between A in (26) and P , both containing decision variables.However, it can be handled by common nonlinear solvers. 2 On the other hand, there are some particular cases in which it can be reformulated as an LMI constraint, as shown in the following section.

A. LMI-Based Control Design
We consider a control system whose overall equations are in the class (4).In this section we show that, in some particular cases, the matrix A of the closed-loop system can be written as A = F + GJ, or as A = F + JG, where F and G are known matrices depending on the system to be controlled, and J is a matrix to be tuned taking the role of the control gain.The objective is to tune J so that the closed-loop system enjoys the δISS property.The following results hold (potentially applicable to both closed-loop or open-loop systems).
Proposition 12: Let us consider a system with equations in the class (4), where A = F + GJ.Let F := W F and G := W G, where W = diag(L p1 , . . ., L pn ) is the diagonal matrix defined in Section III.If ∃P = P T , having the structure required by Theorem 2, and ∃H such that then, if we set J = HP −1 , the system (4) is δISS.Proof: First, note that from ( 27) it follows that P 0 and P −1 0. Second, by resorting to the Schur complement, it holds that Since H = JP , then we can write where A = F + GJ.Note that P −1 = P −T 0 is a matrix with the same structure of P .From (28), the assumptions of Theorem 2 hold, concluding the proof.Proposition 13: Let us consider a system with equations in the class (4), where A = F + JG.Let F = W F , where W = diag(L p1 , . . ., L pn ) is the diagonal matrix defined in Section III and L pi > 0 ∀i.If ∃P = P T , having the structure required by Theorem 2, and ∃H such that then, if we set J = W −1 P −1 H, the system (4) is δISS.Proof: First, note that from (29) it follows that P 0. Second, by resorting to the Schur complement, it holds that Since H = P J, where J := W J, then we can write where A = F + JG.From (30), the assumptions of Theorem 2 hold, concluding the proof.Note that the feasibility problem, which consists of finding P with the required structure and H such that ( 27) or ( 29) Fig. 2. State-feedback control scheme: K is the state-feedback gain matrix, S is the system to be controlled, y s is the output of the system, x s is the state of the system, u 0 is the feedforward term, u K = Kx s , and u s is the manipulated variable.
Now, we will show some examples of control design problems which can be solved using the results in Propositions 12 and 13.Note that, if J is a full matrix whose elements are the controller parameters, then H is a full matrix as well.However, if J is a block matrix containing some zero blocks, then further constraints on the structure of the matrices H and P must be considered, as we will see in some of the following examples.
Example 5. Static linear state-feedback controller: We consider the problem of designing a state-feedback gain matrix K such that the control system in Fig. 2 enjoys δISS.The equations of the system are defined as in (25).In this example, the equation of the controller is where K ∈ R m s ×n s , u 0 ∈ R m s is a suitable feedforward term possibly depending upon the reference signal, and the state x s ∈ R n s is measurable or can be estimated by a suitable observer.Note that the state is certainly known in case it depends only on current and past input and output samples, e.g., a shallow NNARX, where W 0 and b 0 are a priori selected and W 0 is square and invertible.In [26], in case of ESNs, a possible observer was proposed.More generally, some insights about the design of suitable observers for generic systems in class ( 4) are provided in Section V-B.Hence, the closed-loop system has equations in the class (4), where A = A s + B s K has the structure required by Proposition 12 with F = A s , G = B s , and J = K.
Example 6. Echo state dynamic output-feedback controller: We consider the problem of designing an output-feedback controller for the control scheme in Fig. 1, where the system lies in the class (25) and the controller is described by an ESN.We define the equations of the controller as where the direct dependence of the input in the output equation is omitted, i.e., W out 2c = 0 l c ,m c .By recalling that u s (k) = y c (k) and e(k) = r(k) − y s (k), the equations of the overall closed-loop system are in the class (4), where is the overall output, and u(k) = r(k) is the overall input.Since the system matrices are known and the matrices of the controller in the state equation are randomly generated, the only unknown matrix is W out 1c .Hence, it is possible to write Therefore, it is possible to apply the result in Proposition 12.However, in order to obtain a J with the required structure, it is necessary to further constrain 1) the structure of the matrix where H ∈ R l c ×n c is a free variable; and 2) the structure of the matrix P , i.e., P = P T = diag(P 1 , P 2 ), where P 1 ∈ R n c ×n c and P 2 ∈ R n s ×n s have the structure required by Theorem 2.

Example 7. Shallow NNARX dynamic output-feedback controller:
We consider the problem of designing an outputfeedback controller for the control scheme in Fig. 1, where the system lies in the class ( 25) and the controller is described by a shallow NNARX (8).We use the subscript c to denote the matrices and dimensions of the controller.For simplicity, we set b c = 0 ν c ,1 and b 0 c = 0 l c ,1 , which is reasonable in case normalized data are considered.We also assume that W 0 c is a priori selected, whereas and W u c are the controller unknown matrices to be tuned.Note that the controller can be written in the state-space representation (24), where x c (k) is defined as in (10), A c as in (11), . By recalling that u s (k) = y c (k), the equations of the overall closed-loop system are in the class (4), where is the overall output, and u(k) = r(k) is the overall input.Hence, it is possible to write A = F + JG, where order to obtain a J with the required structure, it is necessary to further constrain 1) the structure of the matrix H, i.e., H = [0 T n j 1 ,n j 2 H T ] T , where H ∈ R ν c ×n j 2 is a free variable; and 2) the structure of the matrix P , i.e., P = P T = diag(P 1 , P 2 ), where, in turn, P 1 ∈ R n j 1 ×n j 1 and P 2 ∈ R ν c ×ν c are matrices with the structure required by Theorem 2.

B. Observer Design
In general, if the state is not measurable, the application of state-feedback control schemes (e.g., the one in Fig. 2) requires the availability of a state estimate.For a system in class (4), the following observer is proposed to provide a reliable estimate x of the state x, based on the input-output measures u and y: where L is the observer gain to be designed according to the following Proposition 14: Let us consider a system with equations in the class (4) under Assumption 1.Let us define the diagonal matrix W = diag(L p1 , . . ., L pn ) as specified in Section III.If the observer ( 33) is employed, with L such that Proof: First, note that the dynamics of the estimation error is defined as ê(k) := x(k) − x(k).By jointly considering ( 4) and ( 33), the 2-norm of the estimation error at time instant k + 1 can be written as According to Assumption 1 and to the definition of W , we can write Thus, the condition guarantees that the estimation error converges to 0, i.e., ê(k) → 0 and x(k) → x(k) as k → +∞.Note that ( 35) is equivalent to the following condition: which can be recast as (34) in view of the Schur complement.
The study of the convergence rate of the observer as well as the analysis of the case in which a nonmeasurable disturbance acts on the system state and/or output will be matter of future research.

C. Control Schemes With Zero Steady-State Error
In this section, the possibility to guarantee zero steady-state error in case of tracking of piecewise constant reference signals for systems in the class (4) is investigated.This can be guaranteed, e.g., using the control schemes in Fig. 3, where the system S is in the class (25), the controller C is in the class (24), and the block " " denotes a discrete-time integrator with equation where η ∈ R l s is the state of the integrator, and M ∈ R l s ×l s is its gain matrix.In the following proposition we will prove that all the control schemes depicted in Fig. 3 lead to a common general model of type: where r ∈ R l s is the reference input, y s ∈ R l s is the output of the system, χ ∈ R n χ is a vector of states, and C χ ∈ R l s ×n χ .We can state the following result.Proposition 15: Let the controller C be in the class (24) and the system S be in the class (25).The equations of the closedloop systems in Fig. 3 lie in the class (37).Moreover, the set of systems in the class (37) is a subset of the set of systems in the class (4).
Proof: See the Appendix.
Given an equilibrium point, provided that the control schemes in Fig. 3 are δISS, the zero steady-state error is ensured by the explicit integral action, since such an equilibrium is globally asymptotically stable according to Definition 5.However, there are some cases in which the δISS property cannot be enforced to the control schemes in Fig. 3, e.g., if the output y s of the system is bounded.This is the case of some RNN architectures where all the activation functions are bounded, e.g., the hyperbolic tangent or the sigmoid function.Some examples are ESNs where In this regard, the following result holds.Proposition 16: Let us consider a control system with equations in the class (37).Let at least one output of the system be bounded, i.e., y si (k) ≥ y min and/or y si (k) ≤ y max for all k, and for at least one i = 1, . .., l s , where y max , y min ∈ R.Then, the control system (37) cannot enjoy the δISS property.
Proof: See the Appendix.
Hence, if we consider the system (37), where f χ (•) is composed of bounded nonlinear globally Lipschitz continuous functions, we have that the assumptions of Theorem 2 can never be fulfilled, as a consequence of the result in Proposition 16.Nevertheless, if we consider a system structure where at least a state equation in (37a) is linear, and the latter state directly affects the output in (37c), then y s may be unbounded, and the condition in Theorem 2, as well as δISS, can be enforced.The following example corroborates the statement in Proposition 16 and our remark.
Example 8. State-feedback control with explicit integrator: We consider a SISO system in the class (25) with two states, where f s (•) = [tanh(•) f 2 (•)] T .Let us consider a state-feedback control law and an explicit integral action (36) as in Fig. 3(a), i.e., u s (k) = Kx s (k) + M (η(k) + e(k)), where e(k) = r(k) − C s x s (k).As stated in Proposition 15, the closed-loop system is in the class (37), and by extension also in the class (4).The matrix A of the closed-loop system is and can be rewritten as A = F + GJ, where According to Theorem 2, matrix P must have the following structure: P = diag(p 1 , P 2 ), where p 1 ∈ R and P 2 ∈ R 2×2 .A feasible solution is returned by solving (27) (where W = I 3 ) with YALMIP and MOSEK, ensuring that the closed-loop system enjoys the δISS property according to Proposition 12.The following matrices and parameters are obtained: P =  K = [0.01960.5016], and M = 0.1694.In Fig. 4, the reference tracking results of the closed-loop system are depicted, where we can see that the closed-loop system enjoys δISS and its equilibria are asymptotically stable even if the open-loop system displays unstable dynamics, as we can see from the second (linear) state equation of the open-loop system.Furthermore, due to the explicit integral action, a zero steady-state error is achieved.
On the other hand, if we take f 2 (•) = tanh(•), matrix P is constrained to have the following structure, on the basis of Theorem 2: P = diag(p 1 , p 2 , p 3 ), where p 1 , p 2 , p 3 ∈ R.An unfeasible solution is returned by solving (27) (where W = I 3 ), corroborating the statement in Proposition 16.
If the control schemes in Fig. 3 cannot be used to achieve zero steady-state error while ensuring δISS for the closed-loop system, a possible solution to improve the static performance could be the use of a δISS feedforward compensator in the class (4).Since it is dynamic, this compensator can be used to enhance both static and dynamic performances.Moreover, provided that the closed-loop system is δISS, the addition of a δISS feedforward compensator preserves the δISS of the overall control system.This fact follows straightforwardly from the result in Proposition 10, since such a component is placed in series to the closed-loop system.Future research will address methods for the design of feedforward compensators, as well as the analysis of the dynamic performances of the control system.

VI. SIMULATION RESULTS
In order to validate the theoretical results in the previous sections we propose here a simulation example.The case study consists of the control of the following ESN-based nonlinear SISO system with n s = 8 states: where the direct dependence of the input in the output equation is absent, i.e., W out 2s = 0, and  is the discrete-time integrator, ESN the echo state dynamic controller, and S the system to be controlled.
inclusion of an explicit integral action (according to Proposition 16), and to more freedom in the control design (e.g., see Theorem 2).
The model ( 38) is identified using a noiseless dataset containing 1400 normalized input-output data collected with a sampling time T s = 25 s from a simulated pH neutralization process (see [26], [35], and [36] for a detailed description of the system model).The pH process is a nonlinear SISO dynamical system where the input is the alkaline flowrate, and the output is the pH concentration.The training input data consist of a multilevel pseudorandom signal [35], whose amplitude is in the range [12,16] mL/s.The training is carried out according to the "ESN training algorithm" (see [26] for an accurate description).Basically, W x s , W u s , and W y s are randomly generated, whereas W out 1s is obtained by solving a least squares problem based on the available dataset, where the initial 100 data points are discarded to accommodate the effect of the initial transient.To test the identification performance, the following fitting index is calculated over a validation dataset composed of 600 new normalized input-output data: where y is the real system output sequence, y s is the output sequence obtained with (38), and ȳ is a vector with all the elements equal to the mean value of the real output sequence y.A satisfactory fitting F IT % = 90.2995% is achieved.The control objective in this example is the achievement of perfect asymptotic tracking of constant reference signals.To this aim, the control architecture in Fig. 5 is taken into account, where an explicit integral action is also embedded.In particular, S is the system (38) to be controlled, whose state is assumed measurable, " " is defined as in (36) by setting M = 1, and "ESN " is a non-strictly proper ESN controller with n c = 5 states, whose input vector contains the integrator output v jointly to the system state vector x s .Overall, the controller equations are the following: u s (k) = y c (k) and e(k) = r(k) − y s (k), we can write the closed-loop system equations in the class (4), where the matrix A is defined as , and W y c are known randomly generated matrices, we can rewrite A as F + GJ, where ] takes the role of the control gain.Note that the unknown controller parameters can be computed as [W out 1c W out 2c ] = JE −1 , where According to Proposition 12, the matrix P must have the following structure: P = diag(P D 1 , P F , P D 2 ), where P D 1 ∈ R 5×5 and P D 2 ∈ R 3×3 are diagonal matrices, whereas P F ∈ R 6×6 is a full symmetric matrix.A feasible solution is returned by solving (27) (where W = I n c +n s +1 ) with YALMIP and MOSEK, ensuring that the closed-loop system enjoys the δISS property due to Proposition 12.In Fig. 6, the reference tracking results of the closed-loop system starting from 20 different random initial conditions are depicted, where we can see that the equilibria are asymptotically stable and the output trajectories converge to each other in view of the δISS property.Moreover, due to the explicit integral action, a zero steady-state error is achieved.In Fig. 7, the corresponding control input trajectories are shown.
The previously tuned controller is also tested on the pH process physics-based simulator.To this aim, some remarks  are due.First, in the control scheme, a denormalization of the control variable u s and a normalization of the output y s are performed upstream and downstream of the process, respectively.A normalization of the reference signal is also carried out.The same normalization parameters applied in the identification are employed.
The second issue to be considered concerns the fact that the state measurements of the system (38), necessary for the statefeedback, are not available in this second case.Hence, a suitable observer is required.As suggested in Section V-B, the following observer is tuned: where the observer gain L is designed using the result in Proposition 14.In Fig. 8, the reference tracking results of the closed-loop system using the simulated pH process and starting from 20 different random initial conditions are represented, where we can see that the equilibria are asymptotically stable and, due to the explicit integral action, a zero steady-state error is also achieved.In Fig. 9, the corresponding control input trajectories are depicted.

VII. CONCLUSION
In this article, we have proposed a novel LMI-based condition that guarantees δISS for a class of RNNs.The reduced conservativeness of this condition with respect to other conditions in the literature has been proven.Since this condition is based on LMIs, it is computationally lightweight and can be used for both analysis and control system design.More specifically, we have shown that it is efficiently applicable for the design of dedicated control systems including feedback regulators (in the form, e.g., of static state-feedback controllers, echo state-based, or neural NARX-based output feedback dynamic controllers).The design of observers and the possible inclusion of integrators in the control scheme have been also investigated.Simulation results have corroborated the effectiveness of the theoretical results.
Future work will tackle a number of issues remained open in this work.First, we will address a possible extension to alternative conditions in order to include other classes of RNNs (e.g., multilayers NNARX or long short-term memory networks), as well as a deeper comparison and the extension to control design of the contraction conditions in [22].Second, the development of a (possibly data-based) cost function which takes into account the desired dynamic performances of the control system will be investigated [34].Then, we address the design of a suitable δISS feedforward compensator to be used to achieve static precision in case an explicit integral action cannot be embedded in the control scheme.Furthermore, the convergence rate of the observer together with the analysis of the case in which a nonmeasurable disturbance acts on the system will be studied.Finally, the application of the theoretical results in this article to an experimental apparatus will be carried out.

A. Proof of Proposition 2
First, note that D δ must be invertible in view of Proposition 1. Also, the condition inf D δ ∈D σ(D δ W x D −1 δ ) < 1 Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
is equivalent to require the existence of Now, let us define P = D T δ D δ .Note that P 0 diagonal, if and only if there exists D δ diagonal and invertible such that P = D T δ D δ .Thus, condition (42) (and, in turn, the condition in Proposition 1) is equivalent to require the existence of a diagonal matrix P 0 such that W T x P W x − P ≺ 0 (43) which coincides with the condition in Theorem 2 for system (6a) in case all the activation functions are strictly nonlinear, i.e., W = {1, . . ., ν}, and the Lipschitz constants are L pi = 1, ∀i = 1, . . ., ν.This concludes the proof.

B. Proof of Proposition 4
The objective is to prove that the fulfilment of the assumption of Proposition 3 implies the fulfilment of the assumptions of Theorem 2. The latter, for a single-layer NNARX, require the existence of a matrix P = P T 0 such that A

Furthermore, we can define
, where Therefore, the assumptions of Theorem 2 can be rewritten as follows: W T 0 A T φ P A φ W 0 − P ≺ 0, Now, let us choose a block diagonal P , i.e., P = P T = diag( P 1 , P 2 , P 3 ), where P 1 ∈ R τ ×τ , P 2 ∈ R l×l , and P 3 ∈ R m× m .With this choice, we can compute βL 2 p W T φ W φ − γI (l+ m)N ≺ 0 (47) are fulfilled, then (45) certainly holds.Therefore, we want to prove that there exist α > 0, β > 0, and γ > 0 such that (46) and (47) hold provided that the assumption of Proposition 3 is fulfilled.First, note that (46) holds, if and only if and n j 2 = (m c + l c )N c + m c .Therefore, it is possible to apply the result in Proposition 13.However, in Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

Fig. 3 .
Fig. 3. Closed-loop control schemes with explicit integral action, where is the discrete-time integrator, and S the system to be controlled.(a) Static state-feedback controller with gain K. (b) Integrator in series to the controller C. (c) Integrator in parallel to the controller C.

Fig. 4 .
Fig. 4. Output trajectory of the closed-loop discrete-time system with state-feedback and explicit integral action.Black dashed line: Reference signal trajectory; blue line: Output signal trajectory.

Fig. 5 .
Fig. 5.Control scheme with explicit integral action and ESN in series:is the discrete-time integrator, ESN the echo state dynamic controller, and S the system to be controlled.

Fig. 6 .
Fig. 6.Denormalized output trajectories of the closed-loop discretetime system starting from different initial conditions, with zoom of the initial transient.Black dashed line: Reference trajectory; colored lines: Output trajectories for 20 different initial conditions.

Fig. 7 .
Fig. 7. Denormalized control input trajectories of the closed-loop discrete-time system starting from different initial conditions, with zoom of the initial transient.Colored lines: Input trajectories for 20 different initial conditions.

Fig. 8 .
Fig. 8. Output trajectories of the closed-loop system using the simulated pH process and starting from different initial conditions, with zoom of the initial transient.Black dashed line: Reference trajectory; colored lines: Output trajectories for 20 different initial conditions.

Fig. 9 .
Fig. 9.Control input trajectories of the closed-loop system using the simulated pH process and starting from different initial conditions, with zoom of the initial transient.Colored lines: Input trajectories for 20 different initial conditions.