Control for Multiplicative Noise Systems With Packet Losses and Measurement Delays

This paper mainly considers the control problems for discrete-time multiplicative noise systems with packet losses and measurement delays. The main contributions are two-fold. Firstly, based on Pontryagin’s maximum principle, the optimal output feedback controller is obtained. Secondly, a necessary and sufficient stabilization condition for multiplicative noise systems is derived in terms of the coupled algebraic Riccati equations. Moreover, it has been proved that the stabilization condition only depends on the eigenvalue of the system matrix and the probability of packet losses which is not related to measurement delays.


I. INTRODUCTION
Modern control theory is based on state variables that are whole description for the system and are usually not measurable. Thus the optimal estimator is an important issue in solving problem of the measurement feedback control for networked control systems(NCSs) with packet losses and delays. For stochastic systems with multiplicative noise, in classical estimation problems without control signal, the observation is always assumed to contain the signal to be estimated and minimum mean-square estimator in [1]. When control variables involving in stochastic systems with multiplicative noise, optimal control gains depend not only on the cost function weights, but also on the covariances of the multiplicative noise processes which lead to the optimization problem being not separable in [2]. For linear systems, using uncertain observations, the recursive least-squares state estimators can be given(see [3], [4]). The problem of sliding mode control design for linear systems with incomplete and noisy measurements of the output and additive/multiplicative exogenous disturbances has been studied in [5]. The decentralized optimal control problem [6] for linear discrete-time systems with multiple input channels has been presented.
The associate editor coordinating the review of this manuscript and approving it for publication was Ton Duc Do . The interacting multiple model (IMM) estimator for NCSs with packet losses is developed in [7]. A system comprising sensors that communicate with a remote estimator by way of a so-called collision channel can be traced back to [8].
It should be highlighted that multiplicative noise problems for linear control systems are closely related to the output feedback control and stabilization problems for NCSs where controllers, sensors, and actuators are connected by a digital communication network. Using a shared network is technically and economically preferred to peer-to-peer connection. However, contention based networks may exhibit bandwidth limitations, date packet disordering, limitations arising from data quantization and time-varying sampling period (see [9]- [11]). Time delays are normally caused by signal transmission and traffic congestion, and packet losses are by virtue of nodes failure or long time delays (see [12]- [15]). For nonlinear NCSs with packet losses, there are some developments in virtue of applications of practical control systems. Takagi-Sugeno fuzzy systems have been well recognized as an effective class of models to deal with nonlinear systems problems. In [16], the problem of control synthesis of discrete-time Takagi-Sugeno fuzzy systems is investigated by employing a novel multiinstant homogenous polynomial approach. In [17], the problem of relaxed realtime scheduling stabilization in the Takagi-Sugeno fuzzy VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ model form is obtained by proposing a new alterable-weightsbased ranking switching mechanism. For NCSs, the problems of optimal control and stabilization are getting more and more attention. Recently, the recursive optimal estimator has been derived and the asymptotic stability of error covariance matrix has been investigated in [18]. And the optimal estimator comes to a conclusion that the well celebrated separation principle holds. Based on this work, many studies have been done. The control problem for discrete-time NCSs with packet dropout and input delay is obtained in [19]. But the optimal control problems for multiplicative noise systems are not derived in the above works. Besides, in the distributed NCSs model with asymmetric information, the optimal control and stabilization problems are investigated when the remote device and local device exist simultaneously(see [20], [21]). However, the measurement delays are not considered in those studies, which they are common in the practical application. To the best of our knowledge, there is seldom process on the complete solution to the problem of optimal output feedback control for multiplicative noise systems with both measurement packet losses and delays.
Motivated by this, the output feedback control and the stabilization problems for multiplicative noise systems with measurement packet dropouts and delays will be explored in this paper as shown in Figure 1. Specifically, the arrival of observation data at time k is described by a binary random variable β k with P(β k = 1) = p. The optimal controller is derived by using Pontryagin's maximum principle in [22]. In virtue of a Lyapunov function defined with the optimal cost function, we show that the system is mean square stable if and only if the coupled algebraic Riccati equations (AREs) have a unique specific solution. The rest of this paper is organized as follows. In Section II , the finite-time optimal measurement feedback controller is derived. The infinite horizon control and stabilization problems are considered in Section III . The numerical examples are stated in Section IV . Section V provides the conclusions.
The following notations shall be used throughout this paper.
Notation: R n indicates the n-dimensional Euclidean space, superscript denotes the transpose of a matrix; real matrix A > 0(≥ 0) means A is positive definite (positive semidefinite); L {A} is an indicator function, which means when the element ζ ∈ A, L {A} = 1; otherwise, L {A} = 0. The natural filtration F(y k ) are generated by the measurement process y 0 , . . . , y k . E[·] is the mathematical expectation and E[·|F k ] denotes the conditional expectation with respect to F k ; P(X ) means the probability if the event X occurs; I means the unit matrix, and δ kl indicates the Kronecker delta function, i.e., δ kl = 1 for k = l, otherwise, δ kl = 0.

II. FINITE-HORIZON CASE
A. OPTIMAL FILTERING For multiplicative noise system (2), based on the measurement data {y 0 , . . . , y k }, the optimal filtering is given in the following lemma.
Lemma 1: The optimal estimator is given bŷ where η k = L {y k =0} with L being the indicator function of the set {y k = 0}, P(η k = 0) = p, P(η k = 1) = 1 − p = q and x k−θ +1/k = Fx k−θ/k + Gu k . In addition, the initial condition Proof: The proof of the above lemma is straightforward, thus is omitted here.

B. PROBLEM STATEMENT
This section addresses the problem of optimal measurement feedback control and estimate over NCSs with packet losses and measurement delays. Consider the following system: where x k ∈ R n is the state process, u k ∈ R m is the controller, y k ∈ R n is the measurement process, F,F ∈ R n×n and G,Ḡ ∈ R n×m are deterministic coefficient matrices. Moreover, ζ k is the scalar-valued Gaussian white noise with E(ζ k ζ l ) = σ 2 δ kl .
The associated cost function along with the system (2), (3) is given as: where Q ≥ 0, R > 0, A N +1 ≥ 0. The mathematical expectation E is taken over all the uncorrelated random noises, x 0 , and {β k }. Problem 1: Find the F(y k )-measurement u k such that (4) is minimized subject to (2) and (3).

C. SOLUTION TO PROBLEM 1
We apply Pontryagin's maximum principle to the system (2) with the cost function (4) to yield the following costate equations: where λ k is the costate and Now we define a set of matrix sequences k , H k , K k , A k , B k , by initializing the terminal values A N +1 and B N +1 = 0 and making the following backwards recursion for k = N , N − 1, . . . , θ : In (10), it is assumed that H k is invertible. If this is not the case, the recursion stops. According to above statements, we can give the main results as follows.
Theorem 1: Problem 1 has a unique solution if and only if H k >0 for θ ≤ k ≤ N . If this condition holds, the optimal output feedback controller designed to minimized (4) is given by wherex andx k−θ/k is the optimal estimation satisfying (1). In this case, the optimal cost function is as wherex θ = x θ −x θ/2θ . Besides, the optimal costate λ k−1 and the state x k satisfy the following relationship: Proof: In the first place, the necessity part of the proof is presented as follows.
Necessity: Assuming that Problem 1 admits a unique solution, we will show by induction that H k are strictly positive definite for θ ≤ k ≤ N . Denote For k=N in (18), noting system dynamics (2) and letting x N = 0, the above equation becomes Since J N ≥ 0 and Problem 1 has a unique solution, if there exists H N = 0 such that J N = 0, then u N has numerous solutions contradicting with solution of uniqueness, i.e., H N > 0 can be obtained. Then we shall calculate the optimal controller u N . The equilibrium equation is given as (7), noticing u N is F(y N )-measurement, so we can obtain : Since H N > 0 has been proved, the optimal controller u N can be calculated from (19) as that is, the optimal controller (14) is verified for k = N . Now we shall show that λ N −1 is as (17). In what follows, using (2), (6), (20), the costate λ N −1 satisfies: which implies that (17) holds for k=N.
In order to use the induction method, let any l with θ ≤ l ≤ N , and for j>l + 1, we assume that H j in (10) is strictly positive definite, the optimal feedback controller is presented by (14), A j , B j satisfy the coupled Riccati difference equations (12), (13), and the relationship between λ j−1 and state x j is assumed to satisfy (17).
Further we will show the above statements are also right for j = l.
Firstly, we'll show that if Problem 1 has a unique solution, then H l >0. By virtue of (2), (6), (7), we have Then taking summation on both sides of the above equation from j = l + 1 to j = N yields that Substituting (17) for λ l and setting x l = 0, J (l) can be written as Recalling the case of k = N , we can obtain H l > 0 because of the uniqueness of Problem 1. Secondly, we shall calculate the optimal feedback controller u l , Plugging (17) into (7), it yields that Thus, the optimal controller u l is as Finally, we will show that λ l−1 has the form of (17). Using (2), (8), (17), (22) and given assumption, we have This completes the proof of necessity.
Sufficiency: Given that H k >0 for k>θ , we will verify that Problem 1 admits the unique solution and the optimal cost function is given as in (16).
Now we denote Lyapunov function as where A k , B k satisfy the given Riccati equations (12), (13), respectively. Then we can obtain Taking summation on both sides of (24) from θ to N, it follows that: By substituting the optimal feedback controller as (14) for (25), the minimizing cost function can be calculated as which implies that the optimal cost function has been verified from (26) as (16). This ends the proof. Remark: It should be pointed out that the problems of stabilization can be also solved using the LMIs approach.

III. INFINITE-HORIZON LQR AND STABILIZATION A. PROBLEM STATEMENT
In this section, the infinite horizon measurement feedback control and stabilization problem will be studied.
Consider the following cost function associated with system (2) and measurement process (3): Now we give the following definitions: Definition 1: With controller u k = 0, system (2) is said to be asymptotically stable in the mean square sense, if for any initial state x 0 , the following equation holds: (2) is said to be stabilizable in the mean square sense, if there exists F(y k )-measurement output feedback controller u k = Lx k/k with constant matrix L and x k/k = E [x k |y 0 , . . . , y k ] being the optimal estimation, such that the closed loop system of (2) is asymptotically stable.

Definition 2: System
Definition 3: The stochastic system as below is said to be exact observable, if for any N ≥ θ System (28) is denoted as (F,F, C) for simplicity.
To verify the stabilization problem, we give the following standard assumptions.
Assumption 2: (F,F, Q 1/2 ) is exact observable. In this section, the problem under consideration can be described as follows: Problem 2: Find the F(y k )-measurement output feedback controller u k to stabilize system (2) in the mean square sense, and to minimize infinite horizon cost function (27).

B. SOLUTION TO PROBLEM 2
To be discussing clearly, A k and B k in (12) and (13)  Proof: We have drawn the conclusion that H k is strictly positive definite, thus (12) is solvable. Now we use the induction method to show A k ≥ 0.
For k = N , since A N +1 ≥ 0, B N +1 = 0, Q ≥ 0 and R>0, we can obtain A N ≥ 0. By using induction method, we assume that A k+1 ≥ 0 and B k+1 ≥ 0 for some θ ≤ k ≤ N , then similar to (29) and using (13), there holds A k ≥ 0 and B k ≥ 0, this completes the induction. Hence A k ≥ 0 for θ ≤ k ≤ N has been proved.
Proof: We suppose that if this is not the case, we can assume that for any N > 0, there exists deterministic x (i.e.,x=Ex) satisfying x A θ (N )x = 0.
By choosing the appropriate initial state x 0 to be x( = 0) and u i (i = 0, . . . , θ − 1), it follows that the error covariancẽ x θ = 0 and θ−1 k=0 E(x k Qx k + u k Ru k ) = 0, thus recall from (16) that the minimizing cost function J * N can be presented as where x * k , u * k denote the optimal state trajectory and controller, respectively.
Noticing from Assumption 1 of Q ≥ 0 and R > 0, it follows that: , and u * k = 0. Further, by using the exact observation of Assumption 2, we can obtain x = 0, which contradicts with x = 0.
In conclusion, there exists positive integer N > 0 such that A θ (N ) > 0. This completes the proof.
The main results of this section can be concluded as follows.
Theorem 2: Under Assumptions 1 and 2, multiplicative noise system (2) with measurement process (3) is stabilizable in the mean square sense if and only if max k |λ k ( √ qF)| < 1 and the following coupled AREs admit unique solution satisfying A > 0 and B ≥ 0: where where max k λ k (·) denotes the maximum eigenvalue of a matrix and | · | means the absolute value. In this case, the stabilizing controller is given as withx k/k being the optimal estimation in (15) and H , K given in (32) and (33). Moreover, the stabilizing controller (34) also minimizes the cost function (27), and the minimizing cost function is as In the first place, the necessity of the Theorem 2 is verified as follows.
Necessity: Based on Assumptions 1 and 2, suppose system (2) is stabilizable in the mean square sense, we will show the Riccati equations (30) and (31) admit a unique solution A > 0 and B ≥ 0, and max k |λ k ( √ qF)| < 1.
First, we shall show A θ (N ) and A θ + qB θ (N ) are both bounded. VOLUME 8, 2020 Since it is assumed that there exists output feedback controller u k = Lx k/k with L being constant matrix to be determined such that the close-loop system of (2) satisfying Then we can easily obtain that there exists constant c 1 and Meanwhile, there exists appropriate constant λ 1 , λ 2 such that Q ≤ λ 1 I and L RL ≤ λ 2 I hold with I being the unit matrix. Therefore then we can obtain that is Equation (36) indicates the following. 1) If x θ is deterministic, i.e., x θ = Ex θ andx θ = 0, we have that

then we can obtain that
In conclusion, A θ (N ) and A θ (N )+qB θ (N ) are both bounded.
Next, we will show A θ (N ) and A θ (N ) + qB θ (N ) are both monotonically increasing with respect to N.
In fact, we can easily observe that the optimal cost function given in (16) implies that 2) Assume that x θ is a arbitrary random vector with Ex θ = 0, i.e., E(x θx θ ) = qE(x θ x θ ). In this case, there holds . Now it is evident that A θ (N ) and A θ (N ) + qB θ (N ) are both monotonically increasing with respect to N.
Since the variables given in (10) − (13) are time invariant for N, i.e., In conclusion, we have shown A θ (N ) and B θ (N ) are convergent, i.e., there exists constant matrices A and B satisfying Moreover, from Lemma 4, we know that there exists N 0 > 0 such that A θ > 0 for N > N 0 , thus there holds A = lim N →∞ A θ (N ) > 0.
Similarly, taking limitations on both sides of (9)-(11), the convergence of k (N ), H k (N ), K k (N ) can be obtained Furthermore, it is noted from (12) and (13)  In what follows, we shall show the uniqueness of A > 0 and B ≥ 0.
We will prove by contradiction. Assuming that there exists V > 0 and Z ≥ 0 being another solution to the coupled Riccati difference equations(30), (31), that is For any N, the optimal cost function has been proved to be (16) and notice the convergence of A θ (N ) and B θ (N ), we can obtain that the minimizing cost function is 2)In addition, if Ex θ = 0, i.e., E(x θx θ ) = qE(x θ x θ ), then there holds from (37) that A + qB = V + qZ, thus we have B = Z.
Therefore, the uniqueness of A > 0 and B ≥ 0 has been verified.
Finally, we shall show max k |λ k ( √ qF)| < 1. Suppose there exists some k such that |λ k ( √ qF)| ≥ 1. Actually, we have concluded that for any N, J * N and B θ (N ) are bounded.
Now the sufficiency of the Theorem 2 is verified as follows. Sufficiency: Now we will show the system (2) is mean square stabilizable with controller (34) if the ARES (30), (31) have unique solution A > 0, B ≥ 0, and max k |λ k ( √ qF)| < 1. In order to show that system (2) is mean square stabilizable with controller (34), we define Lyapunov function candidate as From (39) there holds where u k in (34) has been inserted in the last equality of (40). Equation (40) indicates function V (k, x k ) is monotonically decreasing and further noting (39) that V (k, x k ) is bounded below, i.e., V (k, x k ) is convergent with respect to k.
Taking summation on both sides of (40) from θ to N, we can obtain Via a time-shift of l, and taking limitation of l, (41) implies that where the last equality of (42) is implied by the convergence of V (k, x k ).
Based on Assumption 1 and B θ ≥ 0, the optimal cost function given in (35)  Thus there exists N 0 > 0 satisfying A θ (N ) > 0 for any N > N 0 , so lim k→∞ E(x k x k ) can be obtained.
In conclusion, we can obtain that the system (2) is mean square stabilizable with controller (34).
Next,we will show cost function (35) is minimized by the controller (34). VOLUME 8, 2020 From (40) we can obtain Moreover, noting (39) and (42), there holds that Therefore, by taking limitation of N → +∞ on (44), we have that Noticing x l k l=θ are independent of u k for any k ≥ θ, and from (21) we know that H = lim l→∞ H l > 0, therefore we can obtain that the stabilizing controller (34) minimizes the cost function (27), and the minimizing cost function can be given from (45) as (35).
Under the condition of max k |λ k ( √ qF)| < 1, similar to the derivation of (38), it can be easily verified that (31) admits a solution B ≥ 0 and the optimal cost function J * given in (35) is finite. The proof is complete.

IV. NUMERICAL EXAMPLES
In this section, examples are provided to demonstrate the results in Theorem 2 about the stable problem of system (2). Consider system (2)   As shown in Figure 2 and Figure 3, for measurement delays θ = 2, it can be seen lim k→∞ E(x k x k ) = 0 which indicates that the system state is asymptotically mean square stable with the controller designed above.

V. CONCLUSION
This paper aims to analyse the problem of output feedback control and stabilization for multiplicative noise systems with packet dropouts and measurement delays. Firstly, by using Pontryagin's maximum principle, we have shown the optimal output feedback controller and optimal cost function for the finite horizon. Besides, we show that the multiplicative noise system with measurement delay is mean square stabilizable if the given coupled Riccati equations admit a unique solution satisfying A > 0 and B ≥ 0 and the stabilization of the system relies not on the measurement delay but the eigenvalue of the system matrix and the probability of packet dropout. Future research should be devoted to the development of the output feedback control for multiplicative noise systems with packet losses, measurement delays and input delays.