Online Nonaffine Optimal Tracking Control via Direct Heuristic Dynamic Programming

For discrete-time unknown nonlinear dynamic systems, an online optimal tracking control scheme is developed in this paper, which is implemented by direct heuristic dynamic programming (HDP). To start with, a solution procedure is presented for tracking problems of discrete-time nonlinear systems. Then, the basic structure and mechanism of the direct HDP algorithm are described in detail, including the critic network and the action network. Moreover, based on the direct HDP algorithm, the implementation process of neural networks is provided. Finally, the direct HDP algorithm is applied to enable the original system to track the reference trajectory. The effectiveness of the algorithm is proved in solving tracking problems and online optimal tracking control is achieved.