Stochastic optimal controller design for unknown networked control system under TCP

In this paper, stochastic optimal control design has been considered for linear networked control system (NCS) in the presence of network imperfections such as packet losses and random delays. For evaluating the impact of network reliability on controller performance, Transmission Control Protocol (TCP) is considered with NCS. With uncertain system dynamics and unknown network imperfections, proposed stochastic optimal approach, referred to as adaptive dynamics programming (ADP), uses a novel observer and value function estimator to solve the state estimation and infinite horizon optimal regulation of NCS with the TCP protocol. Update laws for tuning the unknown parameters of novel observer and estimator are derived. Stability region of the linear NCS with and without the adaptive observer is given. Lyapunov theory is used to show that all signals are asymptotically stable (AS) in the mean and the estimated control and observed state signals converge to optimal control inputs and actual states of NCS respectively. Simulation results are included to show the effectiveness of the proposed scheme.