Advanced Affine Optimal Tracking Control Through Online Value Iteration and Its Stability Proof

In this paper, to tackle the optimal tracking control problem for discrete-time systems, an advanced online value iteration (VI) algorithm is developed. First, the derivation of traditional VI for the optimal tracking control problem is revisited. Second, the stability condition of traditional VI is established, in order to evaluate the current iterative tracking control policy. Third, based on the concept of attraction domain, the improved tracking control policies under online VI can be obtained by judging the location of the current tracking error. Finally, we prove the stability of the online VI algorithm for the present tracking control problem. A simulation example is displayed to verify the effectiveness of the developed algorithm.