Optimal Control and Stabilization for Networked Systems With Input Delay and Markovian Packet Losses

The linear quadratic regulation (LQR) problem for discrete-time networked control systems (NCSs) is investigated in this article. The difference from most previous works is that input delay and packet losses occur simultaneously in the communication channel connecting the controller to the actuator. Moreover, the data packet dropout is modeled as a time-homogeneous Markov process which poses challenges due to the temporal correlation. The contributions of this article are twofold. First, by applying the maximum principle involving Markov jumps and delay, the linear quadratic optimal control problem in finite horizon is solved and the solution is given in terms of a forward and a backward stochastic difference equations (FBSDEs-M). Second, under a basic assumption, the infinite horizon optimal control problem is solved and the necessary and sufficient condition for mean square stabilization is given in terms of the solutions to coupled algebraic Riccati-type equations with Markov jumps. The presented results are new to the best of our knowledge since there is no existing work that tackles delay and Markovian packet dropouts simultaneously. It is a generalization of the previous work in which the packet dropouts is modeled as an independent identically distributed Bernoulli process.