State-dependent adaptive dynamic programing for a class of continuous-time nonlinear systems

The state-dependent Riccati equation (SDRE) technique can be used to solve optimal control problems for a wide class of nonlinear dynamical systems. In this method, instead of solving a complicated Hamilton-Jacobi-Bellman (HJB) equation, a state-dependent Riccati equation is solved which leads to a suboptimal control law. However, a priori model of the system must be available to apply this technique to the optimal control problem. In this paper, to solve the SDRE without using a priori model of the system, a direct adaptive suboptimal algorithm is proposed. The algorithm, named state-dependent Riccati equation adaptive dynamic programming (SDRE-ADP), is based on a reinforcement learning approach which can be implemented in an online fashion. Like the SDRE technique, the proposed SDRE-ADP can locally asymptotically stabilize the closed-loop system provided that some conditions are satisfied. Application of the proposed algorithm to an autonomous unmanned underwater vehicle (AUV) and a numerical example shows that it can be effectively applied for nonlinear systems.