Modified Algebraic Riccati Equation Closed-Form Stabilizing Solution

A modified discrete-time algebraic Riccati equation (MARE) is a discrete-time algebraic Riccati equation (DARE) for which the quadratic term is weighted by a modifying parameter α. The MARE is known to arise for example when conducting estimation or stabilization of single-input single-output (SISO) systems subject to packet losses in networked control systems (NCSs). In the present paper we characterize the solution to the MARE in closed form that, to the best of the author’s knowledge, is a completely novel result. We then verify the already known critical value of the modifying parameter αc for which the MARE is solvable and propose closed-form expressions for the optimal state feedback gain matrix. We finally present examples to illustrate the obtained contributions.


I. INTRODUCTION
Riccati equations are a recurrent and important feature in many theoretical control design results and have been the subject of study for a long time, including [1] and more recently [2]. When the unique positive semidefinite solution of a Riccati equation is time invariant we then fall back into the particular, albeit ubiquitous, subset known as algebraic Riccati equations (AREs) [3]- [5]. AREs have played a central role in many control design synthesis procedures, [6]- [8], including H 2 optimal control [9] and H ∞ optimal control [10]- [12]. AREs also have played an important role in the dual problem of optimal estimation, and the prime example is the Kalman filter. See, for example, [13]. Notwithstanding the many theoretical implications that are related to solving an ARE, the solution to the ARE itself is usually obtained through numerical algorithms, [14]- [16]. However, in recent years, a modified algebraic Riccati equation (MARE) has been applied to target tracking [17], [18] and networked control systems (NCSs), and a slew of results in recent years have been devoted to its solution [19]- [21].
The motivation to study the MARE and its solution in this work is because it has been observed that the MARE plays a similar key role in NCS problems, as the discrete-time algebraic Riccati equation (DARE) does in classic control. In [22], [23] the MARE appears in the solution of an optimal estimation problem subject to partial or intermittent observations. In [24], the recursively calculated MARE solves the optimal control of Linear Time Invariant (LTI) systems subject to packet losses for a Tranmission Control Protocol (TCP) network. The optimal estimation problem subject to intermittent observations is further studied in [25], [26] and [27]. In particular in [25] the authors provided a lower limit on the probability of packet loss α c for the MARE solution to converge. More recently, MARE-based results can be found for optimal control, [19], [28]; distributed consensus, [29], [30]; distributed estimation, [31], [32]; and stochastic control [33], [34]. The main difficulties currently associated with the state-of-the-art MARE we just described are that there is no clear understanding of the achievable closed loop pole locations in terms of the design parameters, and (since all of these approaches are numerically based) none provide a good starting point for the analysis and design of the resulting solution.
The main contribution of the present work is a complete characterization of the MARE solution as a function of a state matrix A in R n×n and an input matrix b in R n . To the best of the author's knowledge, such characterization is novel. The work in [34] considers the MARE as a constrained ARE and then approximates the MARE solution by means of a convergence argument. See Lemma 1 and Lemma 2 in [34]. Moreover, in order to introduce a Linear Matrix Inequality (LMI) approach, the authors of [34] further relax the identified ARE constraint from an equality to an inequality, which is potentially a second source of approximation on the resulting MARE solution. These are some drawbacks inherent to a numerical approach, such as the LMI approach. More generally, the main advantage and significance of our proposed characterization of the MARE solution over a numerical approach is insight. A numerical approach will conduct brute force search, which might be sufficient for achieving stability. However, our proposed characterization can allow for analysis, and thus insight, and possibly design the control solution, which in turn is necessary when we must achieve performance objectives.
The remainder of this paper is organized as follows: In Section II, we introduce the standing assumptions for the present paper and discuss the closed-loop pole locations induced by the closed-form solution to the MARE. In Section III, we present the main result of this work, that is the characterization of the MARE solution; and discuss a series of extensions to the main result, which include an analytic expression for the resulting optimal state feedback gain matrix. We proceed to illustrate the proposed theoretical results with appropriate examples in Section IV. In Section V, we conclude by presenting our final remarks and possible future research directions.
Terminology: A discrete-time signal is denoted as x(k), where k = 0, 1, 2, · · · ; and its Z-transform as X(z), z ∈ C. We use bold notation to represent a generic matrix A. We denote the element in the i th -row and j th -column of a matrix A as a ij , and equivalently the overall matrix as A = [a ij ]. If a is in C,ā represents its complex conjugate. H is the Hermitian operator, that is A H =Ā T . For a transfer function

II. PRELIMINARIES
In the present section we present the working assumptions and discuss the closed-loop pole locations imposed by the MARE solution when using state-feedback.

A. ASSUMPTIONS
The main assumption considered is for a minimal realization, that is a controllable and observable realization, represented by matrices (A, b, c, 0) of a single-input single-output (SISO) plant model G(z). The eigenvalues ρ i , where i = 1, · · · , n, of A are in C and are generally assumed to be distinct. The distinct eigenvalues assumption can be lifted in exchange for greater complexity in the analytic solution expression. Other assumptions might be stated when required for some specific results, but the above are standing assumptions for the present paper. In general the increased complexity required to treat the MIMO case tends to cloud the insight that can be gained from an analytic solution. See, for example, the MIMO discussion in [35] for the continuous- time ARE. Thus, a numerical approach should generally be preferred for the MIMO case.

B. CLOSED-LOOP POLE LOCATIONS
Consider, for a SISO system, a state-space description given by is the discrete-time input signal in R and y(k) the discretetime output signal in R. The matrix A is in R n×n , b is in R n×1 and c is in R 1×n . As motivation, a typical NCS setting resulting in the appearance of a MARE is given in Figure 1. See [26]. The objective is to achieve meansquared stability (MSS) of an unstable plant model subject to an i.i.d. Bernoulli process, θ(k) ∈ {0, 1}, with the probability of successful sensor measurement transmission given by pr{θ(k) = 1} = α. Given the proposed statespace description the associated discrete-time MARE is then defined by with 0 < α < 1. Both the matrix solution X and the state weight matrix Q are Hermitian, and the control action weight R m > 0. We now proceed to introduce, in the next lemma, two transfer functions that use the matrix solution of (1).

Remark 1.
Note that the value of α in (1) is assumed to be a known parameter (for example, as discussed previously, a known probability of successful transmission). The modifying parameter γ 1 /(1 + γ 1 ) for the first case of MARE presented in [34] when the system under study is SISO (as in here) coincides with α. However, the modifying parameter γ for the second case of the MARE studied in [34] is directly equivalent to the modifying parameter α presented here. Lemma 1. (S reg (z) and L reg (z)). The transfer functions can be equivalently characterized as where s i , ∀i = 1, · · · , n are the closed-loop pole locations imposed by the state-feedback control law solution Proof. From the definition of S reg (z) we have where we replaced A for VΛV −1 with Λ = diag{ρ i } and V, a matrix collecting all the eigenvectors of A. We continue the algebraic manipulation observing that where γ i are the resulting products between the elements of kV and V −1 b. Furthermore, applying the matrix inversion lemma we have where we replaced A − bk for W∆W −1 with ∆ = diag{s i } and W, also a matrix collecting all the eigenvectors of A − bk. This then reveals that with δ i being the resulting products between each element of kW and W −1 b. From Equations (3) and (4), we observe that S reg z has its zeros at each ρ i and, as defined by k, has its poles at each s i . Thus, we have proved the result for S reg (z) in (2). We then observe that L reg (z) = (1 − S reg (z))/S reg (z), from which we readily verify the proposed expression for L reg (z) in (2), which concludes this proof.
In the following lemma we characterize the spectral factorization imposed by the state-feedback control law solution.
Lemma 2. (Spectral Factorization). The closed-loop pole locations s i , ∀i = 1, · · · , n imposed by the state-feedback control law solution u(k) = −kx(k) with matrix gain k = (R m + b T Xb) −1 b T XA satisfy the following equality: and Proof. We invoke a spectral factorization argument, as shown in [36, pp. 99-103], for which we start from the MARE in (1), multiply the left by b T zI −Ā T −1 and multiply the right by We then add and subtract terms zXA, and rearrange terms We further rearrange terms on the left side of the equality and add the weight R m to both sides as We then add and subtract the term and recognize the gain k where feasible. Starting from (9), these steps further result in Finally we further group terms on the left side as (11) and recognize the proposed S reg (z) and L reg (z) transfer functions as We now use Lemma 1 and replace the alternative definitions for S reg (z) and L reg (z) in (12). As z → ∞, we then obtain We then replace (13) within (12) which results in (5), concluding the present proof.
It is well known that the closed-loop pole locations s i are also defined as the eigenvalues of A − bk. However, we use the spectral factorization argument in Lemma 2 because, albeit numerical for most cases, it does not explicitly a priori require the MARE matrix solution X. That is, by means of the proposed spectral factorization argument, we do not need to previously solve the MARE in order to know the closedloop pole locations. We now introduce the expression for the residue factors r i , with i = 1, · · · , n as The residue factors play a role in the analytic solution to the MARE defined in (1), under the assumptions stated in Subsection II-A.

Remark 2.
Note from the definition of the residue factors r i in (14) that a loss of controllability is evident when any ρ i matches any ρ j .
Remark 3. Note that the complex conjugate case, arising from possible second-order factors present in the plant model, is included in the assumption for the eigenvalues of matrix A since complex conjugate pairs are distinct eigenvalues. This is the reason why we have maintained the complex conjugate notation.

Remark 4.
We can rewrite the MARE as Thus, we observe that the solution to the MARE "equalizes" a standard discrete-time ARE (DARE) with a discrete-time Lyapunov equation.

III. ANALYTIC SOLUTION TO THE MARE
We now proceed to characterize the solution to the MARE under study.

Proposition 1. (MARE Analytic Solution)
The analytic solutionX to the MARE in (1) for α c < α < 1 is given byX with M andQ defined as and the residue factors t i defined as where z i is the closed-loop poles induced by an ARE with control action weight R = R m 1 + 1−α α n k=1ρ k s k and the same state matrix weight Q.
Proof. The analytic solutionX = [x ij ] to the MARE with α → 1 and control action weight R is given bŷ where t i defined as in (17). We observe that a MARE with α → 1 is indeed a DARE, and the proof of (19) is reported in [37]. The DARE resultX = [x ij ] will now serve as a stepping stone to obtain the MARE solution for α = 1. Assume that the MARE is solvable for a known value of α. See for example [22] for conditions on its solvability. The DARE solution defined in equation (19) also solves the MARE in (1) if the control action weight R in (18) satisfies By placing (13) into (20), we retrieve the reported R = R m 1 + 1−α α n k=1ρ k s k . We now use (20) to explicitly obtain the solution to the MARE. From (19), we statê We now proceed to factorizeX and obtain Finally, since α + (α − 1)b T Mb ∈ R, we can directly isolateX obtaining the result reported in (15), which concludes the present proof.

An important question that can now be addressed by the characterization of the MARE solution proposed in Proposition 1 is the convergence condition for α. That is, for what values of α is the MARE indeed solvable?
The following corollary answer this question.
where ρ k , k = 1, · · · , n u is the subset of all the n u unstable eigenvalues of A. If the subset of unstable eigenvalues of A is empty, then α c = 0.
Proof. We observe that the denominator of (15) is a linear function of α. Additionally, for α → 1, the MARE converts into a DARE that is solvable for the proposed R and Q. Moreover, for α → 1, the denominator of (15) is positive; thus, if we define α c as the value of α that makes zero the same denominator, that is then α > α c . From the above, we characterize α c as We now observe that as α → α c , thenX tends to diverge and the effect of Q tends to be negligible. As α → α c , M tends to the solution of a DARE with weights R = 1 and Q = 0. Thus, from a similar spectral factorization argument as presented in Lemma 2 but adapted for the aforementioned DARE with Q = 0, we have that z i → 1/ρ i if |ρ i | > 1 or z i → ρ i if |ρ i | ≤ 1 to achieve the flat spectrum imposed by R. We then observe that b T Mb → nu k=1 |ρ k | 2 − 1 , which upon replacement in (24) gives the expression stated in (23). Finally, if all the eigenvalues of A are stable, then b T Mb → 0, which then reports α c = 0, concluding the present proof.
Remark 5. In particular, in [25] the authors provided, for the MARE solution to converge, a lower limit on the probability of packet loss α c = 1 − 1 ρmax , where ρ max is the largest unstable eigenvalue of A. In [27], the proposed result is further refined for higher-order moments of the expected estimation error covariance matrix, but it is still only a function of the largest unstable eigenvalue of A. In light of the result proved in Corollary 1, we have that the critical value reported in [25], [27] is, generally, not a sufficient value if matrix A contains more than one unstable eigenvalue. The result in Corollary 1 agrees with that reported in [26,Theorem 17.1] as the infimal probability p inf for stability based on an MSS feasibility argument over a one-step-delayed TCP-like feedback. A similar expression, as reported in [38], can be obtained using a Mahler measure argument of the square matrix A.
Remark 6. As stated in Proposition 1, as α c then at least one closed-loop pole s i → 1 and, from the proof of Corollary 1, the auxiliary closed-loop poles However, as α → 1, then R → R m ; thus, the values of each s i will tend to be equal to the values of each respective z i . Finally, as s i are the eigenvalues of A − bk, it can be seen (using (20) and the definition of k) that z i are the eigenvalues of A − αbk.
In the proof of Proposition 1 we use the DARE result reported in [37]. However, as stated in (19), this result assumes matrix A is diagonal. To lift this assumption we now consider the general case in which the original state space representation of the plant model is subject to a nonsingular transformation of the state T such thatx = Tx. The state representation in the new state space coordinatesx(k) becomesx where X T solves the MARE in (1) with matrices A replaced by A T = TAT −1 , b replaced by b T = Tb and weights R T = R m and Q T = T −T QT −1 .
Proof. The proof is direct by substitution.
We observe that Lemma 3 can also be used to relax the distinct eigenvalues assumption if A T is a Jordan normal form with a limiting → 0 argument for each repeated eigenvalue. VOLUME 4, 2016 We now move to the optimal state feedback gain matrix, denoted byk T , when subject to a nonsingular transformation T of the state. In particular, if T collects the eigenvectors of matrix A T , then the A matrix will be diagonal, thus generalizing the result in Proposition 1.

Lemma 4. (Transformed Optimal State Feedback Gain
Matrix) GivenX, from Proposition 1 and a nonsingular transformation matrix T, then the optimal state feedback gain matrixk T in the transformed state coordinates is obtained ask wherek is defined aŝ wherek Proof. Observe from (2) that a partial fraction description of L reg (z) results in with the residue factor r i defined as in (14). Furhtermore, from the equivalent definition of S reg (z) in Lemma 1, we have that a partial fraction description of L reg (z) results in Sincek = [k 1i ] and both partial fraction descriptions are equivalent, we obtain the equality proposed in (27). We then substitute A, b andX ink to observe that the optimal state feedback gain matrix in the transformed state coordinates is given as in (26).
As presented, both Lemma 2 and Proposition 1 are analysis tools for a given problem, such as the NCS mean-squared stabilization problem subject to packet dropouts. Nevertheless, any analysis tool can also be used for a better design by, for example, stating the problem of now finding the R m and Q for a given set of closed-loop poles s i and value of α or design the value of α for specific values of k that do not exceed some given value. Furthermore, it can be argued that the results in Proposition 1 and Lemma 3 are not genuinely in closed-form since we first need to obtain the s i and z i closed-loop pole locations numerically by means of Lemma 2. However, as we present in the next lemma, for a selection of the matrix weights involved in the MARE, we can indeed obtain a closed-form without any previous numerical step.

Lemma 5. (Transformed Optimal State Feedback Gain
Matrix for Q = 0) GivenX from Proposition 1 with Q = 0 and a nonsingular transformation matrix T, then the optimal state feedback gain matrixk T in the transformed state coordinates is obtained aŝ wherek is defined aŝ where n u is the number of unstable eigenvalues of A.
Proof. We observe from Equation 20 in Proposition 1 that In this case, for Q = 0, matrixX is also the solution to a minimum energy DARE. From [35,Prop 3] we obtain that Assuming that A contains n u unstable eigenvalues (n u ≤ n), and without any loss of generality that these n u unstable eigenvalues are the first n u eigenvalues of A, then each t i is evaluated as in (17) with z i replaced by 1/ρ i if |ρ i | > 1 or z i replaced by ρ i if i > n u . Upon replacing the induced closed loop eigenvalues, we retrieve (29), and thus (28), which concludes the proof.

IV. EXAMPLES
In this section we present three examples. The first example illustrates the use of Lemma 2. The second example is taken from [22], and it is used to illustrate Proposition 1. Finally, the third example complements the second example by presenting the case of a 3 × 3 A matrix with a mix of stable and unstable eigenvalues.
Furthermore, we set α as 0.8. From Lemma 2, we have that for this example. Furthermore, the left-hand side of (5), upon replacing S reg (z) and L reg (z) as in Lemma 1, is We recognize from (31) that ρ 1 = 1.1 and ρ 2 = 1.25 and stress the fact that Ψ(z) is a function of the closed-loop locations s 1 and s 2 . We now propose an L 2 error function We now provide an example taken from [22, §4] in the context of NCS to highlight the MARE analytic solution from Proposition 1.
Example 2. The proposed example from [22, §4] considers the following matrices: Since the example is oriented to conduct estimations subject to intermittent observations, we explore the values of α ∈ ]α c , 1[ in the dual transformed setting of matrices defined by The unstable eigenvalues of A are ρ 1 = 1.1 and ρ 2 = 1.25; thus, the critical value α c predicted by Corollary 1 is α c = 0.4711. For this example we now introduce the numerical recursive solution to the MARE defined as We compute the above equation for 1000 iterations and initial numerical recursive value X 0 = 2.2204·10 −16 0 2.2204·10 −16 . We use (36) to obtain the closed-loop pole locations s 1 (α) and s 2 (α) as the eigenvalues of A − c Tk . We also use (36) to obtain the closed-loop pole locations z 1 (α) and z 2 (α) as the eigenvalues of A − αc Tk . Both are reported in Figure 3. The upper panel of Figure 3 shows that as α → α c = 0.4711, the closed-loop pole s 1 → 1 and thus results in instability. Furthermore, as α → α c , we also observe from the same figure that z 1 → 1/ρ 1 and z 2 → 1/ρ 2 , as stated in Remark 6.
From Proposition 1 and Lemma 3, the MARE analytic solution for this example is then defined aŝ with T as defined in (35). In Figure 4, we compare the determinant of the MARE analytic solutionX, solid line, and the MARE numerical solution obtained using the recursive expression defined in (36), dash-dotted line. As we can observe, the agreement is excellent. Finally, from Lemma 4, we can provide the transformed optimal state feedback gain matrix, which results in with again T as in (35).
In the next example, we explore the case of a mix of stable and unstable eigenvalues for a 3 × 3 A matrix.
To evaluate the precision of the MARE solution expressed in (38) we propose the following expression as an error function: where X 1000 is the numerical solution adapting for this example the recursion defined in the previous example in Equation (36), up to a recursion of 1000.
In Figure 5, we report Ξ as a function of α in Db. We observe again that the agreement between the analytic solution X and the numerical solution X 1000 is quite good; thus, the identified analytic solution is indeed the MARE solution for this example.
We observe, as predicted by Corollary 1, that there is a lower limit α c = 1 − 1 ρ 2 1 ρ 2 2 = 0.833 for which the solution no longer converges (vertical black dashed line in Figure 5). Since for this example we considered the choice of Q = 0, we then have that z 1 = 1/ρ 1 , z 2 = 1/ρ 2 and z 3 = ρ 3 . In addition, from Lemma 5, we have that the optimal state feedback gain matrix is then k = 1 α −0.8326 1.2412/0 , since, for this example, A is diagonal and we are assuming T = I.

V. CONCLUSION
In the present work we characterized the closed-loop pole locations induced by the solution to the MARE, through a spectral factorization argument. With this we have proposed a closed-form solution to the MARE and have proved, in a novel way, the existence of a critical value α c for which, as long as α > α c , the MARE solution is guaranteed to converge. We then have focused on a closed-form for the stated feedback optimal gain matrix for the general case, and the specific case in which Q = 0. We have illustrated these results with three examples. Future research based on the present work should consider the case of more structurally complex MIMO systems, and the possible application of the MARE closed-form solution to relevant H 2 or H ∞ control design problems.