Constrained Model Predictive Control for Nonlinear Markov Jump System With Persistent Disturbance via Quadratic Boundedness

In this article, a robust quadratic-boundedness-based model predictive control (MPC) scheme, for a discrete-time nonlinear Markov jump system (MJS), is extended to the case with persistent bounded disturbance and nonhomogeneous transition probability. By applying the S-procedure, the constraint conditions, the persistent bounded disturbance and the sufficient stability conditions are all derived in term of a few linear matrix inequalities (LMIs), thus the original min-max optimization problem is transformed into a convex optimization problem in LMI paradigm. At each sampling time, the control moves satisfying the control constraint are obtained online and implemented in the nonlinear MJS. Quadratically boundedness and min-max MPC are combined to achieve the closed-loop stochastic stability of the controller with respect to the persistent bounded disturbance. A numerical example is presented to demonstrate the effectiveness of the proposed results.


I. INTRODUCTION
In some research areas, such as biochemical system, economic system and energy system, the structures and parameters of engineering systems would vary abruptly due to sensor or actuator failures, disturbance inputs, sudden environmental changes, economic scenarios, or temporary loss of communication between subsystems [1]. Markov jump systems (MJSs), a well-known class of stochastic switching systems, were found to represent these random variations much more suitable for over two decades of research. In general MJSs, containing various influencing factors of stability, such as nonlinearity, uncertainty, time-varying, time-delay, disturbances and strong constraints, are a set of dynamic with the switching among the modes controlled by a Markov chain [2], [3]. The control problems of MJSs have attracted much attention since 1960s. Many interesting results for MJSs can be found in the literature, such as sliding mode control [4]- [6], neural network control [7], fuzzy control [8], [9], The associate editor coordinating the review of this manuscript and approving it for publication was Bing Li . state estimation [10]- [13], H 2 /H ∞ optimal control [14]- [16] and model predictive control [17]- [24], etc.
The adaptive sliding mode controllers based on the sliding surface and the switched Lyapunov function were investigated to guarantee the stochastic stability of MJSs with respected to time-varying actuator faults, partly unknown transition probabilities, unknown matched nonlinearity or unknown external disturbances [4]- [6]. With multilayer neural network and delay-independent conditions, a mode-depended finite-time controller was designed to make the MJSs stochastic stabilizable with both Markovian jumping parameters and mixed time delays [7]. Extended the conclusions presented by [8], a fuzzy controller was designed for nonlinear MJSs with general unknown transition probabilities based on the Takagi-Sugeno fuzzy models in [9]. When states of MJSs are unmeasured, the state estimation scheme is a good choice. By employing the high-gain scaling technique, common Lyapunov function method and backstepping technique, a novel reduced-order dynamic gain observer and an output-feedback risk-sensitive control scheme was designed in [10], which can eliminate the stringent assumption in [11]. By using full information estimation and moving horizon VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ estimation, the fixed-horizon sliding algorithm based on the Bayesian networks theorem can take more account of both the constraint conditions and the nonlinearity than other state estimation algorithms [12], [13]. In [14], a state-feedback control scheme was analyzed for stabilization, H 2 and H ∞ performance under a hidden-observation which recovered many relevant cases preciously studied in the literature of MJSs, such as clustered observations, detector-based observations and periodic observation of the mode signal. Because of the excellent capability to balance system performance and robustness, mixed H 2 /H ∞ optimal control was successfully applied in MJSs with time-delay [15], input constraints and disturbances [16]. Model predictive control (MPC) is a powerful optimization control strategy for nonlinear constraint system because it is feasible to handle hard constraints on the operated and controlled variables in a systematic manner during the design of controller by solving linear matrix equality (LMI) [16], [25]- [27]. In the process of rolling optimization, containing the nonlinearity, strong constraints and disturbances, the objective function can be solved online to obtain the control moves or state feedback gain matrices, which would steer the states into an ellipsoid invariant set or a polyhedral invariant set. In recent years, MPC was applied widely in linear MJSs, as presented in [17]- [19]. The stability and feasibility of a tree-based MPC optimization was guaranteed as well as the full-scenario linear MJSs in [20]. Based on the periodic invariant set, a feedback predictive control method can reduce more conservativeness than that of using a modedependent feedback control law [21]. In [22], N suboptimal controllers were designed offline and stored in a look-up table. By looking up the table online, the control law can drive the state of MJSs to a neighbor area including the origin.
Although much achievement of MPC had been made for linear MJSs, most real systems are essentially nonlinear. There is an increased interest in the study of nonlinear model predictive control (NMPC) based on the nonlinear predictive models. Thus the optimization problems of NMPC are nonlinear and nonconvex, which are difficult to solve even for cases involving only few variables [23]. If the nonlinear items can be represented by neural network model [28], T-S fuzzy model [29]- [31] or polyhedral model [23], [32], [33], the linear analytical expressions of original systems can be obtained and the relatively mature results on stability and feasibility of linear MPC can be applied in nonlinear MJSs. Considering the nonlinear MJS with nonhomogeneous process, the constrained MPC design was proposed and avoids solving nonlinear optimization problem through applying a differential-inclusion-based design [23]. Lots of practical systems are often subjected to disturbances, especially bounded persistent disturbances, which will deteriorate the control performance and stability of the systems drastically. For state-delayed MJSs with exponential decay disturbance, sliding mode controllers can guarantee the closed-loop stochastic stability [5], [6]. For linear MJSs subjected to the linear combinational constraints of states and input controls, constrained MPC synthesis based on coupled invariant sets can guaranteed the convergence of the closed loop responses [19], [24]. Considered input-to-state stable problems and Lyapunov-like sufficient conditions, MPC scheme was a powerful tool for nonlinear systems subjected to hard constraints and disturbances [28], [29], [32], [33]. Using quadratic boundedness, the receding-horizon estimators were designed for linear systems with bounded disturbances [34], [35].
In this article, we aim to investigate a robust MPC design problem for nonlinear constrained MJSs with persistent bounded disturbance via quadratic boundedness approach. The main contributions are listed as follows: first, motivated by the approach in [33], the bounded persistent disturbance can be involved in a finite horizon optimal problem by applying the S-procedure. The LMI-based MPC controller online optimizes a control move for the nonlinear MJSs, which can restrict the states into a robust invariant set. Second, inspired by [34] and [35] the stable constraint condition is established to guarantee the quadratic boundedness and the stochastic stability. A key technique for this MPC scheme is an appropriate formulation of the disturbance which accounts for recursive feasibility of the optimization problem.
This article is organized as follows. In Section 2, the problem setup and some preliminary results are presented. Section 3 provides the main results including modedependent MPC design and the stochastic stability analysis. Finally, an illustrative numerical example and some conclusions are given in Sections 4 and 5, respectively.
Notations: In the sequel, for a positive-definite matrix P > 0 and a vector x, x 2 P = x T Px. E{·} denotes the mathematical statistical expectation of a stochastic process or vector. The expression λ max (P) and λ min (P) denote the maximal and the minimal eigenvalue of P separately. Prob{·} is the probability measure of an event.

II. PROBLEM STATEMENT AND PRELIMINARY RESULTS
Consider a constrained discrete-time nonlinear MJS with bounded persistent disturbance given below: where x k ∈ R n x , u k ∈ R n u and ω k ∈ R n ω are the state vector, control input vector and disturbance vector respectively. The persistent disturbance is subjected to The mode r k is the discrete-time Markov stochastic process taking values from a finite state set S = {1, 2, · · · , M }. The states and the input control of the system (1) are subjected to the following box constrains: where i = 1, 2, · · · , n x , j = 1, 2, · · · , n u , x and u are given constant vectors.
Assuming that the nonlinear function F(·) is continuous differentiable at the origin and the equilibrium of the system (1) is F (r k , 0, 0, 0) = 0. In the neighborhood of the equilibrium point of the system (1), a polytopic description system including original system can be constructed by use of the Taylor series extension and the extreme value of partial differentiation [32] or the differential inclusion [23]. L vertices are assumed to vary in the set Moreover, there exist parameters α l (k) ≥ 0 such that L l=1 α l (k) = 1 and If (k) is a constant matrix, the system (1) reduces to a homogeneous MJS. Assuming that the time varying transition matrix (k) has H vertices, h (k) denotes the vertex h of the transition probability matrix (k). Hence, the time varying transition matrix of MJS (1) is constructed as The polytopic description of the system (1) is described by A mode-dependent state feedback control input is defined as where F k (r k ) is the state feedback gain matrix for mode r k at time k.
, then the closed-loop MJS can be described by We aim to design a MPC controller to stabilise the system (1) with the persistent disturbance (2) in the quadratically boundedness and mean square sense, while guaranteeing constraints (3) and optimising the state responses of the closedloop system (9). Definitions and lemmas of various stochastic stability concepts for MJSs are presented next.
Definition 1 [23]: For an initial state x 0 and any initial mode r 0 , the discrete-time MJS (9) is said to be stochastically Definition 2: For any disturbance ω k ∈ P ω (k ≥ 0), if there exist γ (k) ≥ 0 and a set of symmetric positive definite matricesP k+1 and P k (r k ) satisfied the stable constraint the closed-loop MJS (9) is quadratically bounded with stochastical Lyapunov matrix P k (r k ), where P k (r k ) is the stochastical Lyapunov matrix for mode r k at time k,P k+1 is the stochastical Lyapunov matrix for mode r k+1 at time k +1, Alessandri et al. in [34] proposed quadratic boundedness to deal with stability and design of receding-horizon estimators for linear system. Based on quadratic boundedness, Ding and Pan in [35] developed an output feedback robust MPC design for linear polytopic uncertain system with bounded disturbance. Now, we extend this result to the nonlinear MJSs.
where M 1 and M 2 are real matrices and N is a positive matrix of compatible dimensions. Lemma 2: Suppose that there exists a symmetric positive definite matrix P k (r k ) such that then the closed-loop MJS (9) with ω ≡ 0 is stochastically stable.

III. ONLINE MODE-DEPENDENT MPC DESIGN
In this section, a robust online MPC scheme will be constructed to minimize the performance function J (k). Let u k be calculated online as u k+0|k , and the future optimal control input be u k+n|k for n = 1, 2, · · · , N , where N is the length of predictive horizon. where R . Q > 0 and R > 0 are the weighted matrices of compatible dimensions. Let us define the Lyapunov function for n = 1, 2, · · · , N , where P k+n (r k+n ) ∈ R n x is the symmetric positive definite matrices for mode r k+n at time k + n. Suppose that MJS (9) satisfies the stable constraint Summing the inequality (19) from n = 1 to ∞, we can get the upper bound of J ∞ 1 (k) From Lemma 1, we have where µ k+1 = λ max (P k+1 )/λ min (P ω ), In order to guarantee stochastic stability of MJS (9), we impose, at each sample instant k In (24), γ (k) is an upper bound as follows: Theorem 1: For MJS (9) subjected to constraints (2), (3), (24) and (25), take Q k+n (r k+n ) = P −1 k+n (r k+n ), Y k+n (r k+n ) = F k+n (r k+n )Q k+n (r k+n ), P ω = γ (k)P ω and Q k+n+1 = P −1 k+n+1 . Then (17) can be solved by the following optimization problem min u k ,γ (k),Q k+1 ,µ k+1 ,Q k+n (r k+n ),Y k+n (r k+n ),Q k+n+1 ,P ω γ (k) where r k , r k+1 ∈ S , l ∈ S , h ∈ S , n = 1, 2, · · · , N , µ k+1 = µ −1 k+1 , η ∈ (0, 1) is a suitable scalar. Proof: Applying the Schur complements to inequality (25), we have Applying the Schur complements again, the above inequality is equivalent to and we can obtain inequality (27).
Since the disturbance ω satisfies the constraint (2), the inequality V (x k+n|k , r k+n ) ≥ γ (k) is equivalent to V (x k+n|k , r k+n ) ≥ γ (k)ω T k+n|k P ω ω k+n|k for all ω T k+n|k P ω ω k+n|k ≤ 1. By applying the S-procedure, it is shown that (24) is equivalent to (31) and is also equivalent to Apply the quadratic function (18) to rewrite (32) as where l (r k+n ) = A l (r k+n ) + B l (r k+n )F k+n (r k+n ), By using the Schur complements, inequality (34) can be rewritten as and is also equivalent to By using the Schur complements again, we can get inequality (28).
Theorem 2: If the optimization problem (26) has a solution at time k, MJS (9) is quadratically bounded with stochastical Lyapunov matrix P k (r k ). The set = {x k |x T k P k (r k )x k ≤ γ (k)} is a robust invariant ellipsoid of MJS (9) and (26) is feasible at time k + 1.
Proof: From Theorem 2, MJS (9) is quadratically bounded with stochastical Lyapunov matrix P k (r k ), which means that Substituting (9) with ω k ≡ 0 into (51), we can obtain Multiplying (52) by the coefficients β h (k) to sum from h = 1 to H , we can obtain where From Lemma 2, MJS (9) with ω k ≡ 0 is stochastically stable. If x T k P k (r k )x k ≤ γ (k), the set = {x k |x T k P k (r k )x k ≤ γ (k)} is a robust invariant set of MJS (9). Thus, x k → 0 as k → ∞.

IV. ILLUSTRATIVE NUMERICAL EXAMPLE
Consider the discrete-time nonlinear MJS with two modes (M = 2) [23], Mode 1: Mode 2: The control input constraint |u k | ≤ 4 and the state constraints |x k,i | ≤ 2, i = 1, 2 are imposed for all k. Assuming the equilibrium of the system is the origin, the nonlinear MJSs can be described by polytopic description as follows [23]:  In the simulation the parameters are Q = diag{1, 1}, R = 1, N = 5, r 0 = 1, ε = 0.1, η = 0.1. Solving the optimal problem (50) online, the simulation results are depicted in the following figures. Compared with the algorithm in [23],   the proposed Algorithm 1 in this article gets a larger terminal region as shown in Figure 1. For an initial states x 0 = [1.8, 1.9] and ω k = 0, the comparison of the state responses and control inputs is shown in Figure 2. Algorithm 1 steers VOLUME 8, 2020  Figure 4 and 5. Although the initial states don't satisfy the state constraints, the closed-loop states can converge into the terminal regions, so the proposed MPC controller has a strong robustness.

V. CONCLUSION
In this article, we have proposed a new robust MPC design method for nonlinear MJSs with persistent bounded disturbance. Based on double polytopic descriptions of the nonlinear MJSs and transition probability matrices, the persistent bounded disturbances are involved in the stability condition via quadratic boundedness and all constraint conditions are converted into a convex optimization problem described by some LMIs. Solving the optimization problem at each sampling instant, the control move can guarantee the states of MJSs converge into the terminal region. The simulation results have demonstrated the effectiveness and performance of the proposed method. It appears that the methodology is extend to low online computation burden and low conservation of the ellipsoidal terminal region.