Online Value Iteration for Discrete-Time Nonlinear Optimal Regulation with Stability Guarantee

In this paper, the intelligent and online value iteration (VI) algorithms are developed to solve the optimal control problem for nonlinear discrete-time systems. First, the intelligent VI algorithm combines the advantages of traditional VI initialized by the zero cost function and stabilizing VI initialized by the admissible control policy. The traditional VI is easy to implement and can provide the initial admissible control policy for the stabilizing VI. Meanwhile, stabilizing VI can guarantee all control policies are admissible. Second, based on the concept of the attraction domain, an online value iteration algorithm is proposed to regulate the closed-loop system by using immature control policies rather than the fixed optimal control policy. It ensures that the state trajectory converges to the origin of the attraction domain. Finally, simulations are carried out and the results show the effectiveness of the two new VI algorithms.