Online adaptive data-driven control for unknown nonlinear systems with constrained-input

In this paper, we discuss a novel algorithm for learning the solution to the optimal control problem (OCP) for affine nonlinear continuous-time constrained-input systems with completely unknown dynamics on a data-driven integral reinforcement learning (IRL) basis. It is well known that we have to obtain the solution of the nonlinear OCP by means of resolving the Hamilton-Jacobi-Bellman equation (HJBE). However, the HJBE is usually a nonlinear partial differential equation that cannot be solved analytically. To make matters worse, most practical systems are too complex to be accurately mathematically modelled and have real-time errors in the system’s controller. To address the above issues, we propose an online data-driven IRL algorithm that is anchored in policy iteration (PI), using real-time data from practical systems, rather than system models or partially sampled data from systems. To begin with, the PI algorithm is shown. Then, we approximate the performance function and the control policy using a critic neural network (CNN) and an actor neural network (ANN), respectively. The approach presented is an online-policy IRL, where the data are continuously sampled in the input and state domains. The weights of the CNN and ANN are renewed by least squares using the collected data, which minimizes residual errors. Finally, the validity of the approach in solving the OCP is demonstrated from the simulation results.