ADP-Based Optimal Control of Linear Singularly Perturbed Systems With Uncertain Dynamics: A Two-Stage Value Iteration Method

We study the problem of adaptive dynamic programming (ADP) based on optimal control of linear singularly perturbed systems (SPSs) subject to completely unknown dynamics. Previous works on ADP-based optimal control of SPSs require that the fast dynamics of the system are either known a priori or unknown but asymptotically stable, and such results are limited to standard cases, i.e., the internal dynamics of their fast subsystems are nonsingular. In this brief, these conditions can be removed through a sequential procedure to design feedback gains of both fast and slow states in the framework of ADP. In this procedure, two optimal control problems are formulated for the prefeedback fast subsystem and the modified slow subsystem. Then, a data-driven two-stage value iteration algorithm is imposed to learn the optimal controller without using any system dynamics. Fundamentally different from existing works, an initial admissible control policy is no longer needed during learning. Finally, the effectiveness of the learning algorithm is proved by a simulation example.