An Output Feedback Predictive Control for Stochastic Stabilization of a System With Multiple Fixed State Delays and Multiple Markov Input Delays

This paper studies the stochastic stabilization with an output feedback predictive control for a linear system with multiple fixed state delays and multiple Markov input delays. The two equivalent systems are developed through two steps. The original system is transformed to a system with augmented states in a state delay free form. Then, it is further transformed into a system with a predictive state in an input delay free form. The stochastic stability condition of the equivalent system is shown to be a sufficient condition for the stochastic stability of the original system. The stability condition is given as a set of linear matrix inequalities of which matrix size depends only on the maximum state delay, not on the maximum input delay. Simulation results show that the proposed methods provide similar or better performance than the conventional method while their complexities are much smaller in most cases.


I. INTRODUCTION
With the evolution of the internet, everything is being connected. Widespread use of networks has made cyber-physical systems (CPS) and associated control methods more important than ever [1]. The evolution of artificial intelligence also accelerates this trend, since machines or devices can operate autonomously without human intervention from using sensors and data transmitted over a network [2]. However, the transmission of data over the network is bound to have a delay. It can be due to a communication protocol, network congestion, a channel erasure, and etc. While a delay can be harnessed to stabilization with articulated modeling [3], ignoring delay can be very detrimental to some control systems [4].
Stabilization of a control system with delay often uses the Lyapunov-Krasovskii method or Lyapunov-Razumikhin method, which often results in bilinear matrix inequalities (BMIs) or linear matrix inequalities (LMIs) conditions [5].
The associate editor coordinating the review of this manuscript and approving it for publication was Guangdeng Zong .
Existing researches have extended these two approaches through some transformation or inequalities. The stability condition of a linear retarded and neutral type system was given as a set of LMIs with a descriptor form which transformed the differential terms of states into equivalent algebraic terms with delay [6], [7]. Wirtinger's inequality [8] and integral inequality [9] were exploited to derive a less conservative sufficient condition for the stabilization of a system with a delay. Robust control for a system with a Markov delay was developed with practical consideration of quantization [10]. A more extensive review on the stability of the system with delays can be found in [5], [11] and reference therein. It can be found that many existing works have been executed to provide less conservative stability condition for a time-delay system under the various different conditions on delays and uncertainties.
Even though many researches assume that a delay is a fixed constant, a random delay is likely to be a more practical model especially in a networked control system (NCS) [12], [13]. In consideration of time coherence in a communication channel, the random delay is often modeled as a Markov process [14]. A conventional approach for the stabilization of a system with a Markov delay transforms it into an equivalent Markovian jump linear system (MJLS) with augmented states in a delay-free form of which dynamic system matrix follows a Markov process [15]. A Markovian jump linear system is a dynamical system whose state matrix has multiple modes driven by a Makov process. The study of this system is known to be tracked back to an early work on a Makovian jump linear quadratic (JLQ) control [16]. A necessary and sufficient for the stabilizability of a Markovian jump linear system was derived for a continuous time homogeneous Markov process [17], and a discrete time Markov process having a periodic transition matrix [18]. Many existing second moment stabilities for the continuous time jump linear systems were shown to be equivalent [17]. However, it is usually very difficult to obtain the transition probabilities completely. To deal with this problem, the stability condition for MJLSs with partially known transition probabilities were established for a class of a continuous time and discrete system [19]. In addition to the uncertainty due to the partial knowledge of the transition probabilities, disturbance was also considered to make the control for MJLS resilient. The estimation of disturbance was used by an anti-disturbance resilient controller to achieve the stability for a system with both the partially unknown transition probabilities and multiple disturbances without white noise [20] and with white noise [21].
The stability condition in the presence of the Markov delay is often derived with mean square stability criterion rather than deterministic asymptotic stability which can be too conservative. The necessary and sufficient condition for the mean square stability for a system with bounded random input delay was provided in [22]. The sufficient condition for the stochastic stabilization of a system with Markov delay in the presence of uncertainty in some elements of a transition matrix was given as a set of LMIs [23]. An optimal linear quadratic Gaussian (LQG) control for a system with a Markov delay was shown to be decomposed into an optimal estate estimator and a linear quadratic regulator (LQR) with state feedback [24].
Even though a significant amount of research has dealt with a control system with a single delay, a system with multiple delays will be a more general model for which a controller design can not be trivially extended from the single delay case. A stability region for a linear system with two delays was fully characterized through the quasipolynomial characteristic equation in the Laplace domain [25]. The Lyapunov-Krasovskii method was exploited to derive a delay independent sufficient condition for asymptotic stability in the presence of multiple delays [26]. A sufficient and necessary condition for the stability of a system with multiple delays was developed from the extended Farkas lemma and Ramzumikhin stability condition to deal with uncertainties on delay and system [27]. Research on the stabilization of a system with multiple delays has been further extended to a system with both state delays and input delays which is a more general model. A sufficient condition for the stabilization of a linear system was given with the assumption that a state delay was the same as an input delay [28]. The Lyapunov-Krasovskii(LK) functional and integral inequality was exploited to derive a delay dependent stabilization condition for a state feedback control system when a state delay and an input delay were time varying [29].
One of the efficient ways to deal with information absent in a control system due to a delay is to predict the information from the previously available information. A predictive control exploits this intuition by transforming a control system with control input delays into an equivalent system in a delay-free form. Historically, a Smith predictor was first proposed to deal with a pure delay in a closed loop system [30]. A control law called a generalized minimum variance (GMV) method was developed from exploiting the prediction of the output with a single known dead time [31]. It was more refined through generalized predictive control (GPC) in a form of receding horizon control, which took multiple costs and a set of predictions into account [32]. Stability with the predictive control was established for various conditions. A linear predictive feedback control for a system with distributed input delays was shown to achieve exponential stability [33]. A control for a time varying linear system was developed through double transformation to achieve exponential stability [34]. The predictive control was further extended to make it robust to uncertainties. A polytopic uncertainty was added to make a predictive control robust to the uncertainty in fractional delay due to the discretization [35]. Arstein transformation was exploited to design a robust control to uncertainties in system parameter and delay [36]. A predictive feedback for a system with a time varying input delay was developed with a back-stepping approach to make it robust to disturbance [37]. However, many existing literatures are limited to a continuous time system with a single input delay.
In this paper, two predictive controls for a system with multiple fixed state delays and multiple Markov delays are developed from the sufficient conditions for the stabilization of equivalent systems without a delay. Despite its generality of the system model and efficiency of the predictive control, similar existing works cannot be found to the best of the author's knowledge. To derive the proposed predictive control methods, the conventional equivalent system model with state augmentation is exploited first to make it without state delays. Two novel transformations are introduced to make a final equivalent system in a delay-free form for each method. The proposed methods are derived from the sufficient condition for stochastic stability. The proposed methods are shown to achieve stabilization better than the conventional state augmentation method with much smaller complexities.
This paper is organized as follows. In section-II, a description of the considered system is provided. A conventional approach to the equivalent Markovian jump linear system without delay is also given as a baseline method. In section-III, two novel predictive controls to achieve VOLUME 8, 2020 stochastic stability are presented with some details on the implementation procedure. The proposed methods are compared with the baseline approach through numerical evaluations in section-IV. Concluding remarks are made in section-V.

II. SYSTEM MODEL AND PROBLEM FORMULATION
A discrete time linear system with state delays and input delays which can be regarded as a general retarded system can be written as is the bth sensor output, d a is the ath constant state delay, r b,k is a random delay from the bth sensor to the controller unit, of which maximum is r max , M is the number of state delays, N is the number of control inputs, and A a ∈ R n×n , ∀a, B b ∈ R n×m , ∀b, and C b ∈ R p×n , ∀b are constant matrices. It is assumed that d a is indexed in increasing order for the sake of the simplicity of representation. It is also assumed that the minimum delay at each channel is 1, and r b,k follows the first order Markov process. Communication delays have been often modeled using a Markov process [10]. This model can naturally include random delay, channel erasure, and out-oforder packet arrival under the assumption that the most recent data is used by a controller and the decrement of a delay from the arrival of a new packet [14]. In addition, time delays are usually correlated with the last time delays [22]. Moreover, a physical transmission channel is time-correlated especially in a wireless network. The stationary transition probability of a delay is defined by where i ∈ D(r max ) = {1, 2, · · · , r max }. With the assumption that the controller uses the most recent data, the delay at each channel can increase at most by 1, which means However, there are N independent transition at each instance. Let r 1,k r 2,k · · · r N ,k be denoted byr k .r k can jump from r C tor F at time k +1 with probability which can be expressed as where [·] b is the bth element of the vector in the bracket, where K b is static output feedback controller for the bth output, a conventional equivalent state equation in a delay-free form can be given bȳ , 0 and I are zero matrix and identity matrix with proper dimension respectively, and δ(·) is a dirac-delta function of which value is 1 when the argument inside the bracket is 0, otherwise 0. (6) is none other than a Markovian jump system.
To deal with the randomness in input delays, the stochastic stabilization is considered. The definition of the stochastic stability for the clarity of presentation [18], [22] is reproduced as follows.
Definition 2.1: A Markovian jump linear system is said to be stochastically stable if for any initial statex(0) and any ini- 2 |x c (0),r(0)} < +∞ There are several different definitions of stochastic stability. However, it was shown that ''asymptotically mean square stable'', ''exponentially mean square stable'', and ''stochastically stable'' were equivalent [17]. This definition simply implies that the state of a stochastically stable system asymptotically converges to the origin in the mean square sense [38].
The stochastic stability condition for (6) can be represented asĀ where G is a set of N dimensional vector of which element belongs to D(r max ), P(r C ) ∈ S (d max +1)n×(d max +1)n is a positive definite matrix, and P(r C ) is E{P(r k+1 )|r k =r C } which can be expressed as The complexity of this approach heavily depends on d max and r max . In addition, the large matrix size can incur numerical problems. Thus, a method with reduced complexity through predictive control will be developed throughout this paper.

III. STOCHASTIC STABILIZATION WITH A PREDICTIVE CONTROL
When there is an input delay in a control system, the predictive control has been shown to successfully reduce the complexity for finding a control gain [35], [39]. However, it cannot be trivially extended to a system with multiple state delays and multiple input delays, since the multiple state delays cannot be directly transformed into a predictive form and some of the delayed inputs in the existing predictive control may not be available in a causal system. Thus, through more involved procedures and novel transformation, we will develop two predictive controls for a system with multiple state delays and multiple input delays.

A. THE FIRST METHOD
To make the system equation more manageable, (1) can be transformed into a system with augmented states such that resulting state equation has state-delay free form as follows.
x (11) is a system with multiple input delays, we can adopt the procedure presented in [40] to make it delay-free form for a predictive control. We first define N auxiliary states as follows.
where l ∈ D(N ). From (13), z l (k + 1) can be expressed as This can be done by insertingx(k + 1) in (11) into z l (k + 1) in (13) and rearranging them. (14) still has delayed input signals. However, it can be canceled out through summing them all, which results in the following equation.
v(k) will converge to zero when all z b (k) converge to zero. Thus, the control input u b (k) is given as a function of z b (k) as if z b (k) is a system ouput in the following way.
After inserting (17) into (16), the condition for the stochastic stability of v(k) can be easily derived from Lyapunov stability condition as However, what we need is the stochastic stability ofx(k). The following theorem says that the stochastic stability condition for v(k) guarantees the stochastic stability ofx(k).
Theorem 3.1: A system with multiple state delays and multiple Markov input delays in (1) stochastically converges with static output feedback when there exist P b (i) > 0 and K b for ∀b ∈ D(N ) and ∀i ∈ D(r max ) satisfying (19).
Proof: From the triangular inequality and (13), the following inequality holds.
(20) can be further simplified with exploiting eigenvalues of The inequality still holds with squaring each side.
After applying the inequality that the algebraic mean is greater than the geometric mean to the last term in (22), summing over k from 0 to ∞ results in the following inequality, which proves the theorem.
where χ is a finite positive constant. χ is finite since z b (k) is stochastically stable when (19) holds and u b (k) has stochastic stability from (17) when z b (k) has stochastic stability.
Finally u b (k) in (17) can be written in terms of y b (k) as (24) It is noted that the control input depends on the history of sensor output due to the state delays and history of control input due to the input delays.

B. THE SECOND METHOD
The proposed method in the previous section tends to be conservative since it tries to stabilize the effective state v(k) through stabilizing each component auxiliary state z b (k) independently. Since this is associated with remaining delayed terms for each z b (k) after transformation, another transformation to get over this issue is to be developed. The augmented state equation (11) can be represented as Even though each element inū(k − r max ) has a different delay, (25) is now in a standard form for a system with single input delay, which makes it possible to apply the well known standard procedure for the predictive control. We first define an auxiliary state as follows.
With the same procedure in the previous section, z (k) can be written as With control inputū(k) =KCz (k), (28) can be arranged as The stochastic stability condition for z(k) can be easily derived as where A(K ) = (Ā +Ā −r max ( N b=1B bKbCb )) and P (i) = r max j=1 π ij P (j). It is noted that as N increases, there is more chance to find a feasible solution of (30) through combininḡ K b properly since the number of constraints and the matrix size of LMI do not change with the different number of control inputs. However, what we need is the stochastic stability ofx(k) as in the previous section. The following theorem says that the stochastic stability condition for z (k) guarantees the stochastic stability ofx(k).
Theorem 3.2: A system with multiple state delays and multiple Markov input delays in (1) stochastically converges with static output feedback when there exist P (i) > 0 andK b for ∀i ∈ D(r max ) satisfying (30).
Proof: From the triangular inequality and (27), the following inequality holds. Since each element inū(k) has a different delay, u b (k) needs to be expressed explicitly. It can be easily arranged as (32) There are several things to be noted regarding the proposed algorithms. The difference between the two methods originates from deriving an alternative state equation in a delay free form. The first method is to define a state which is the sum of auxiliary states so that the defined state can be in a delay free form. The second method transforms the multiple inputs to a large single aggregated input so that the standard procedure for the predictive control can be applied. Even though control inputs in (24) and (32) have a similar structure, each control signal uses the output signals and the input signals having different delays. It is also noted that the number of constraints associated with the first method is marginally larger than one with the second method.

C. ALGORITHM IMPLEMENTATION
When there are multiple inputs to the system, the system can be stabilized by the number of control inputs less than N . This is going to be critical in the implementation of the conventional approach. In addition, a large number of variables or matrix size can incur numerical problems. Thus, the sequential approach with increasing complexity will be desirable. To this end, we explain some implementation details regarding the proposed algorithm to compare their complexities in a more consistent way.
The pseudo-code for the conventional method is presented in figure-1. It simply starts with the first control input only and sequentially increases the number of inputs. It is noted that regardless of the number of inputs, the matrix size for LMI does not change. The pseudo-code for the proposed methods are summarized in figure-2 and figure-3. They also sequentially increase the number of control inputs to be used while the matrix size for LMI does not change.

IV. NUMERICAL SIMULATIONS
To evaluate the performance of the proposed methods, a hypothetical linear system was considered for n = 2, m = 2 and p = 2. The elements of the matrices A a , B b and C b were generated from a standard normal distribution independently. π ij was set to be 0.5 if i = j, otherwise, 0.5/(r max − 1). 300 realizations were generated to calculate average processing time and the success rates of finding a stabilizing controller as a performance measure. The conventional VK algorithm [41] was used to solve LMIs with the maximum number of iterations, 100. The system state trajectory and control input trajectory follow the equation (1) and the equation (24) or (32) respectively. Depending on a used algorithm, the static output controllerK b are calculated from LMIs presented in figure-1, figure-2, or figure-3.
The performance with increasing r max was shown in terms of the success rates for finding a stabilizing controller in figure-4. The conventional method described in section-II was adopted as a method for baseline performance. The system was configured with M = 2, d 1 = 1, d 2 = 2 and N = 2. All methods are shown to provide consistent performance over the different maximum input delays. Due to the excessive computational time, performance with larger maximum input delay could not be assessed. It is also observed that the second proposed algorithm provides marginally better performance. It may be attributed to the smaller number of constraints and the structural advantage in the constraints. The figure-5 shows that the proposed methods have a significant reduction in computational complexity compared to the conventional method as r max increases. The matrix size of the conventional method for stability condition is (d max + 1)n × (d max + 1)n while it is (d M + 1)n × (d M + 1)n for the proposed methods. Since d max = d M with r max = 2, all the methods have similar complexities. However, as r max increases, in addition to the matrix size, the number of LMIs associated with the stability condition for the conventional method increases too, since it is r N max . Thus, the complexity of the conventional method grows significantly with increasing r max . It is also noted that the complexity of the second method is always smaller than that of the first method. Having a larger number of LMIs associated with the stability condition, the first method may have slow convergence with the VK algorithm or require both control inputs for stabilization more often.
In figure-5, the complexity of the conventional method was shown to be comparable to that of the proposed methods when the maximum input delay is small. It is because the state delays were fixed to be 1 and 2. Thus, the effect of state delay was studied with r max = 6, N = 2, and M = 1 in figure-6. It shows the success rates of finding a stabilizing controller as d 1 increases from 1 to 5. All methods were shown to provide consistent performance for the state delay ranging from 1 to 5. The second proposed method again provides marginally better performance with a similar reason as in the previous case. The average processing time for designing the corresponding controller was shown in figure-7. The advantage of the proposed methods in computational complexity VOLUME 8, 2020   over the conventional method can be clearly observed. Since d max is 6 when r max is 6 and d 1 < r max , the conventional method has significantly larger complexity than the proposed method of which complexity is proportional to d 1 .

V. CONCLUSIONS
In this paper, we proposed two methods for designing a predictive output feedback controller to achieve stochastic stability. Both of the proposed methods used the following steps; it first transformed the system into a system without state delay through state augmentation and then transformed it to a system with the predictive state so that the equivalent system cannot have a delay. Since the matrix size associated with LMIs for stochastic stability condition depends only on the maximum state delay, the proposed system can have significant computational advantage when the maximum input delay is larger than the maximum state delay. Even in the opposite case, the proposed methods are expected to have smaller complexities, since the number of constraints associated with the proposed methods is smaller in most cases. Simulation results confirm that the proposed methods have much smaller complexity in most considered cases while its performance is almost the same as that of the conventional method or even better.
Many issues associated with the considered system configuration remain to be resolved. Even though the state delays were assumed to be known and fixed, they can be time-varying and unknown. Depending on the level of uncertainty, the proper consideration needs to be taken into the proposed method. Another further research direction is to consider model uncertainty. In the presence of model uncertainty, the stochastic stability condition needs to be identified first. Then, controllers optimized for a particular criterion can be developed further. Predictive control may be extended to find a control method for the multi-objective control of a system with multiple delays. It can be also considered to find a distributed control such as consensus control where multiple delays can naturally occur due to the physical configurations of a network consisting of nodes connected with multiple neighbor nodes.