Design and Validation of Zeroing Neural Network to Solve Time-Varying Algebraic Riccati Equation

Many control problems require solving the algebraic Riccati equation (ARE). Previous studies have focused more on solving the time-invariant ARE than on solving the time-varying ARE (TVARE). This paper proposes a typical recurrent neural network called zeroing neural network (ZNN) to determine the solution of TVARE. Specifically, the ZNN model, which is formulated as an implicit dynamic equation, is developed by defining an indefinite error function and using an exponential decay formula. Then, such a model is theoretically analyzed and proven to be effective in solving the TVARE. Computer simulation results with two examples also validate the efficacy of the proposed ZNN model.


I. INTRODUCTION
In control areas, a typical nonlinear matrix equation termed algebraic Riccati equation (ARE) is frequently encountered and needs to be solved accurately [1]- [7]. At present, AREs with the continuous-and discrete-time forms are playing a remarkable role in many engineering applications of control problems [7]- [11], such as the linear-quadratic-Gaussian and H ∞ control problems. The solution of ARE is generally utilized as an essential part of the solution to the aforementioned control problems. For example, the positive definite solution of ARE has been used in each iteration of the homotopy algorithms for the fixed-architecture control [12]. The negative definite solution of ARE has been utilized in the Lyapunov equation approach to solve the matrix differential Riccati equation involved in the linear quadratic optimal control [13]. Thus, an effective algorithm/model for solving the ARE is worth designing and investigating due to its important role.
Several direct algorithms have been reported to solve the continuous-and discrete-time AREs [1]- [4], [14]- [19], such as the Schur and the generalized eigenvalue algorithms. However, computational disadvantages exist for the direct algorithms in several situations [20]. Different iterative algorithms for solving AREs that differ from the direct algorithms The associate editor coordinating the review of this manuscript and approving it for publication was Giambattista Gruosso .
have been designed and investigated [4], [8]- [11], [20]- [25]. For example, the Kleinman algorithm, which is an effective Newton algorithm, has been developed to solve the continuous-time ARE arising in H 2 control [21]. The Lanzon algorithm has been provided for solving the continuous-time ARE arising in H ∞ control [20]. An accelerated iterative algorithm has been proposed and studied in [8] to obtain the positive definite solution of the discrete-time ARE. Such algorithm can also be used to compute the negative definite solution of the discrete-time ARE (once it exists). In addition to such iterative algorithms, the neurocomputing models have been developed to find the (positive/negative definite) solution of ARE [26], [27]. Notably, these algorithms and models have been established intrinsically to solve the time-invariant AREs, where ''time-invariant'' indicates that the coefficient matrices of ARE are constant.
In [28], the time-varying ARE (TVARE) was mentioned and presented for the first time, of which the positive definite time-varying solution was used to design the feedback controller for slowly varying linear system. However, to the best of the authors' knowledge, nearly no research result on seeking the solution of TVARE exists to date. That is, the study of solving TVARE is rare despite its importance in the control of time-varying systems. In general, the TVARE can be treated as a time-invariant ARE within a small time under the assumption of short-time invariance. Then, the aforementioned algorithms and models can be used to compute the solution of TVARE at each single time instant. However, this method of solving TVARE may fail to perform efficiently. In other words, the effectiveness of these algorithms and models in solving the TVARE cannot be guaranteed. Thus, developing and investigating an effective algorithm/model to solve the TVARE directly are imperative.
In this paper, motivated by the inspiring work [28] and considering the advances in neural network [29]- [31], we provide a new and efficient model to determine the solution of TVARE. Specifically, with the definition of an indefinite error function and the usage of an exponential decay formula [29]- [31], a typical recurrent neural network called zeroing neural network (ZNN) formulated into an implicit dynamic equation is developed and studied. Then, the convergence property of the proposed ZNN model is theoretically analyzed and proven. Specifically, the model can effectively solve the TVARE. Computer simulation results with two examples also validate the efficacy of the proposed ZNN model.
The remainder of the paper is organized as follows. Section II presents the preliminary of solving the TVARE. Section III describes the proposed ZNN model for solving the TVARE. Theoretical analyses on the model convergence are also discussed. Section IV shows the simulation results by using the proposed model. Section V provides the final remarks. The main contributions of this study are summarized and presented as follows.
1) The ZNN model is proposed and studied for solving the TVARE. The proposed model has not been reported and investigated in the existing literature. 2) This paper theoretically analyzes and proves that the proposed ZNN model can generate an exact solution of TVARE. Computer simulations are also conducted to demonstrate the efficacy of the proposed model. 3) This paper is the first attempt to provide an effective model for computing the solution of the TVARE. This improvement is crucial because it shows a potential for designing different models to solve additional (timeinvariant and/or time-varying) nonlinear matrix equations/inequalities.

II. PROBLEM STATEMENT
Consider the following time-varying linear system [28]: where u(t) ∈ R m is the input or control vector and x(t) ∈ R n is the state vector withẋ(t) as its time derivative. The coefficient matrices A(t) ∈ R n×n and B(t) ∈ R n×m are assumed to be bounded at any time instant t 0. In addition, their time derivatives, i.e.,Ȧ(t) andḂ(t), are bounded at time t 0.
In [28], the state feedback control of the system (1) is investigated by considering the control u(t) = −B T (t)P(t)x(t), where P(t) ∈ R n×n is the positive definite solution of the TVARE as follows: where C(t) = A(t) + γ I ∈ R n×n , D(t) = B(t)B T (t) ∈ R n×n , and Q(t) ∈ R n×n is a symmetric matrix. In addition, γ > 0 ∈ R is the given parameter, and I ∈ R n×n is the identity matrix. Notably, coefficient matrices C(t), D(t), and Q(t) in (2) vary very slowly unlike the dynamics of the time-varying linear system (1).
With regard to the TVARE (2), the set of time-varying solutions can be either finite or infinite [4]. For presentation convenience, we denote P * (t) ∈ R n×n as a theoretical time-varying solution of (2), andṖ * (t) as its time derivative. To guarantee the existence of P * (t) andṖ * (t), the following solvability assumption of (2) is presented [4].
Solvability Assumption: The time-varying coefficient matrices D(t) and Q(t) in the TVARE (2) satisfy at any time instant t 0.
In this paper, our objective is to develop a new neurodynamic model for determining an exact time-varying solution of the TVARE (2) under the assumption of (3).

III. MAIN RESULTS
To solve the TVARE (2), the indefinite error function E(t) ∈ R n×n that must converge to zero is defined as follows: To make E(t) be convergent to zero, the following decay formula [29]- [31] is utilized: whereĖ(t) is the time derivative of E(t), λ > 0 ∈ R is a design parameter and (·) : R n×n → R n×n is an activation function array. Notably, each element of (·), i.e., φ(·), is a monotonically increasing odd function, such as the linear, power-sigmoid, or hyperbolic-sine function [29]- [31]. Substituting (4) into (5) yields the result as follows: which is further changed to the following formulation: withĊ(t),Ḋ(t),Ṗ(t), andQ(t) as time derivatives of C(t), D(t), P(t), and Q(t), respectively. Let us define the following matrices:  (7) for solving the TVARE (2).
with symbol ⊗ being the Kronecker product [32]. In addition, let vectors y(t) and z(t) be y(t) = vec(P(t)) ∈ R n 2 and z(t) = vec(Q(t)) ∈ R n 2 withẏ(t) andż(t) corresponding to their time derivatives, where vec(·) denotes the vectorization operator for a matrix. On the basis of the Kronecker-product and vectorization technique [32], the dynamic equation (6) is thus reformulated as follows: where F(·) : R n 2 → R n 2 denotes the vectorization of (·) and is also a monotonically increasing odd function array. Hereafter, (7) is termed the ZNN model that is proposed in this paper for solving the TVARE (2). For better understanding on the structure, the proposed ZNN model (7) is presented in the following ith neuron form: where ψ = n 2 , y i is the ith (with i = 1, 2, · · · , ψ) element of (7) and f (·) is a processing element of F(·). The weights ik , ξ kj , and ω kj represent the ikth element of the inverse of (M (t) + N (t)), the kjth element of (U (t) − V (t)), and the kjth element of ((M (t)+W (t)). The thresholdsż k and z k represent the kth elements ofż(t) and z(t). Therefore, the structure of (7) is shown in Fig. 1.
Theorem: Given a solvable TVARE (2), the neural state of the proposed ZNN model (7) using a suitable activation function array is convergent to an exact solution of (2).
Proof: For the TVARE (2), let us define P * (t) ∈ R n×n as a theoretical solution, which satisfies the following equation: To lay a basis for further discussion, let matrices G(t) and (9) withṖ * (t) andĠ * (t) as time derivatives of P * (t) and G * (t).
As analyzed above,Ẽ(t) → 0 with t → +∞. The following result is thus obtained: This result denotes that the state matrix P(t) of (10) will converge to a theoretical solution P * (t) of the TVARE (2). That is, P(t) → P * (t) as time t evolves. Notably, the vectorization form of (10) is exactly the proposed ZNN model (7), wherein the state vector y(t) is the vectorization of P(t); i.e., y(t) = vec(P(t)). By summarizing the above results, it is concluded that the neural state of the proposed ZNN model (7) with appropriate activation function array is convergent to an exact solution of the TVARE (2). The proof is thus completed. The above theorem theoretically guarantees that the proposed ZNN model (7) is effective in solving the TVARE (2). The following propositions are presented to further show the convergence characteristic of (7), where S + ∈ R n×n denotes a given positive definite matrix and S − ∈ R n×n denotes a given negative definite matrix.
1) In the case of selecting the linear activation function array, the exponential convergence with rate λ is achieved for (7), which corresponds to the exponential convergence of P(t) to P * (t) of (2). 2) In the case of selecting the hyperbolic-sine, power-sum, or power-sigmoid activation function array, superior convergence is achieved for (7) in comparison with the linearly-activated ZNN model. Remarks: The selection of the initial state y(0) is important to the computation of the proposed ZNN model (7), leading to different computational results of (7). It follows from Proposition 1 that y(0) can be selected according to different situations. For instance, to obtain the positive definite solution of the TVARE (2) involved in the control of slowly-varying systems [28], we can simply set y(0) to be y(0) = vec(I ) (i.e., the vectorization of the identity matrix), or directly set y(0) to be y(0) = vec(P * (0)), where P * (0) is determined by using the MATLAB routine ''CARE'' to solve (2) at t = 0 s. Besides, it follows from Proposition 2 that the activation function has a significant effect on the convergence characteristic of (7). In the ensuing simulation part, the linear, hyperbolic-sine, power-sum, and power-sigmoid activation functions presented in previous studies [29]- [31] are employed for investigating the proposed ZNN model (7) to solve the TVARE (2). As a matter of fact, by referring to [33]- [35], designing more different types of activation functions for further enhancing the convergence performance of (7) can be a future research direction.

IV. SIMULATION VALIDATION
In this section, simulation results are presented to validate the efficacy of the proposed ZNN model (7) for TVARE solving. The simulations are performed by using MATLAB R2008a on a digital computer with an Intel(R) Core(TM) i3-3110M @2.40 GHz CPU, 4 GB memory, and Windows 7 OS.
Example 1: In this example, the TVARE (2) is considered with the following matrices: − cos(t) sin(t) − cos(t) 5 + sin(t) , where d 11 (t) = (4 + exp(−t) − cos(t)) 2 , d 22 (t) = (2 + 1/(t + 1) + sin(t)) 2 , q 11 (t) = 3 + (1/(t + 1)) 2 + sin(t), and q 22 (t) = 6 + (2 + exp(−t)) 2 − cos(t). This situation corresponds to the time-varying linear system (1) with the coefficients as follows: − cos(t) sin(t) , To solve the TVARE (2) with above matrices, the proposed ZNN model (7) is simulated and studied. The related simulation results are shown in Figs. 2 through 5. Figure 2 presents the results of solving the TVARE (2) via the proposed ZNN model (7) using λ = 1 and linear activation function array and starting from six initial states selected as y(0) = vec(S + ). In the figure, the residual error  is defined as e(t) 2 = (M (t) + W (t))y(t) + z(t) 2 = C T (t)P(t) + P(t)C(t) − P(t)D(t)P(t) + Q(t) F with · 2 and · F corresponding to the two norms and the Frobenius norm. As shown in Fig. 2(a), all the state trajectories of (7) converge to a time-varying solution. Fig. 2(b) denotes that such a solution is positive definite in terms of its eigenvalues being greater than zero. In addition, as seen from Fig. 2(c), all the residual errors of (7) are convergent to zero, thereby indicating that the positive definite solution computed by (7) satisfies C T (t)P(t) + P(t)C(t) − P(t)D(t)P(t) + Q(t) = 0. These simulation results verify that the proposed ZNN model (7) with y(0) = vec(S + ) can generate an exact positive definite solution of the TVARE (2). Figure 3 shows the results of solving the TVARE (2) via the proposed ZNN model (7) using λ = 1 and linear activation function array and starting from six initial states selected as y(0) = vec(S − ). As presented in Fig. 3(a) and (b), all the state trajectories of (7) are convergent to a time-varying negative definite solution (as its eigenvalues are smaller than zero). Fig. 3(c) indicates that all the residual errors of (7) converge to zero, thereby showing that the negative definite solution computed by (7) also satisfies C T (t)P(t)+P(t)C(t)− P(t)D(t)P(t) + Q(t) = 0. These simulation results verify that the proposed ZNN model (7) with y(0) = vec(S − ) can generate an exact negative definite solution of the TVARE (2). Figure 4 illustrates the results of solving the TVARE (2) via the proposed ZNN model (7) by increasing the value of λ (i.e., from 1 to 10). As seen from the figure, 1) starting from y(0) = vec(S + ), the state trajectory of (7) converges to the positive definite solution; and 2) starting from y(0) = vec(S − ), the state trajectory of (7) converges to the negative definite solution. These phenomena coincide with the theoretical results given in Proposition 1. Notably, the solutions at steady-state in Fig. 4 are the same as those in Figs. 2(a) and 3(a), thus meaning that such solutions computed by (7) also make C T (t)P(t) + P(t)C(t) − P(t)D(t)P(t) + Q(t) = 0 hold true. Fig. 4 further shows that all the residual errors of (7) converge more rapidly, and the steady-state errors are in the orders of 10 −5 and 10 −6 . These simulation results indicate that the proposed ZNN model (7) is effective in solving the TVARE (2). Moreover, by comparing Fig. 4 with Figs. 2 and 3, we find that, with the increase of λ, the convergence performance of (7) will be enhanced (in terms of its convergence time being shortened by increasing the λ value). Thus, for the proposed ZNN model (7), λ should be selected as a sufficiently large value to meet the practical requirement.   (7) using λ = 10 and linear activation function array and starting from twelve randomly-generated initial states, with the fist six initial states selected as y (0) = vec(S + ) and the rest selected as y (0) = vec(S − ).
For further investigation, the proposed ZNN model (7) is simulated by using y(0) = [1, 0, 0, 2] T , different values of λ, and different activation function arrays. The related simulation results are presented in Fig. 5, and they denote that (7) effectively solves the TVARE (2) in terms of the convergence of residual errors. Moreover, from Figs. 2 through 5, the observations are perceived as follows, which coincide with the theoretical results given in Proposition 2.
• The convergence time for (7) is shortened by 10 times as λ increases by 10 times, thus denoting that (7) possesses the property of exponential convergence.
• Increasing λ effectively enhances the convergence performance of (7). thus indicating again the importance of λ in (7) for TVARE solving.
• By selecting the hyperbolic-sine, power-sum, and power-sigmoid activation function arrays, (7) possesses better convergence that the linearly-activated model. On the basis of these observations, increasing the value of λ and selecting a suitable activation function array result in superior performance of (7). In summary, Figs. 2 through 5 verify the efficacy of the proposed ZNN model (7) for solving the TVARE (2).
Example 2: When the coefficients in (2) are the constant ones, the TVARE reduces to a time-invariant ARE. In this example, the proposed ZNN model (7) is investigated to solve the TVARE with the following constant matrices [27]: For solving such a (time-invariant) ARE, the MATLAB routine ''CARE'' is employed, and the solution is obtained as follows: By using and simulating the proposed ZNN model (7) to solve the TVARE (2) with above constant matrices (Notably, in this situation,Ċ(t) =Ḋ(t) =Q(t) = 0), the simulation results are presented in Figs. 6 and 7. Figure 6 shows the results of solving the time-invariant ARE (i.e., (2) with constant matrices) via the proposed ZNN model (7) using λ = 1 and linear activation function array and starting from the initial state selected as y(0) = vec(0.5I ). As shown in Fig. 6(a) and (b), the state trajectory of (7) is convergent to the positive definite solution (as its eigenvalue is greater than zero). Furthermore, the residual error in Fig. 6(c) converges to zero, showing that the solution in Fig. 6(a) satisfies C T (t)P(t) + P(t)C(t) − P(t)D(t)P(t) + Q(t) = 0. Notably, the solution computed by (7) at t = 10 s is given as follows: Thus, the error between vec(P care ) and vec(P neuro ) is obtained as follows: where symbol | · | returns to the absolute value for each element of the vector. This small error indicates that the solution via the MATLAB routine ''CARE'' and the solution via the proposed ZNN model (7) are almost the same as each other. These results verify the efficacy of (7) for solving the time-invariant ARE (being a special case of the TVARE (2)). Figure 7 presents the results of solving the time-invariant ARE via the proposed ZNN model (7) by changing the value of λ from 1 to 10, which denotes the model effectiveness for time-invariant ARE solving. That is, the state trajectory of (7) converges to the positive definite solution rapidly and the corresponding residual error converges to zero. In addition, comparing Fig. 6 with Fig. 7 shows that the convergence performance of (7) is enhanced with the increase of λ. Besides, the following solution computed by (7) at t = 10 s is This solution is quite similar to the solution via the MATLAB routine ''CARE'', with their small difference/error being presented as follows: Evidently, this error is smaller than the previous one, i.e., 10 −8 versus 10 −5 . This means that by increasing the value of λ, the time-invariant ARE can be solved more accurately via the proposed ZNN model (7). These results indicate again that the proposed ZNN model (7) is effective in solving the time-invariant ARE and that the design parameter λ, which should be selected as a sufficiently large value, is an important factor affecting the performance of (7).
In summary, Figs. 6 and 7 verify that the proposed ZNN model (7) can generate an exact positive definite solution of the time-invariant ARE, further showing the efficacy of (7) for solving the TVARE (2).

V. CONCLUSION
In this paper, the ZNN model (7) was proposed and studied to solve the TVARE (2). Such a model which is depicted in an implicit dynamics and employs the derivative information of coefficients was theoretically proven to be capable of generating an exact solution of (2). Simulation results further verified the efficacy of the proposed ZNN model (7) for TVARE solving. By following this paper, the proposed ZNN model (7) will be further investigated for the control of slowly-varying systems [28]. Besides, designing a new ZNN model for solving the TVARE (2) in noisy environments [31] will be another future research direction.