Online Single-Network Algorithm to Solve the Model-Free Finite-Horizon Optimal Tracking Control Problem

In this paper, an online completely unknown nonlinear continuous-time system based on neural network (NN) is proposed, and then a novel formulation of the value function is given to solve the finite-horizon optimal tracking control problem (OTCP), which avoids the construction of traditional augmented system,. First, using available input and output data, a recursive neural network (RNN) is used to build a data-driven model to reconstruct the unknown system. Then, based on the reconstructed system model, the time-varying Hamilton-Jacobi-Bellman (HJB) equation is established according to the proposed value function. Since the HJB equation is time-dependent, the time-dependent activation function of NN is considered. In addition, an additional error is introduced to satisfy the terminal constraint of the value function. This adaptive control algorithm can approximate the optimal value function in real time and guarantee the Ultimately uniformly bounded (UUB) for the closed-loop system. Finally, a simulation example is used to verify the effectiveness of the proposed control scheme.