A Physics-Informed Neural Networks Framework to Solve the Infinite-Horizon Optimal Control Problem

In this work, we leverage physics-informed neural networks (PINNs) to approximately solve the infinite-horizon optimal control problem for nonlinear systems. Specifically, since PINNs are generally able to solve a class of partial differential equations, they can be employed to approximate the value function in the infinite-horizon optimal control problem, via solving the associated steady-state Hamilton-Jacobi-Bellman (HJB) equation. However, the issue with such a direct approach is that the steady HJB equation generally yields more than one solution, hence directly employing PINNs to solve it can lead to divergence of the method. To tackle this problem, we instead apply PINNs to a finite-horizon variant of the steady-state HJB equation which has a unique solution, and which uniformly approximates the infinite-horizon optimal value function as the horizon increases. A method to verify whether the selected horizon is large enough is also provided, as well as an algorithm to increase it with reduced computations if it is not. Unlike conventional methods, the proposed approach does not require knowledge of a stabilizing controller, the execution of computationally expensive iterations, or polynomial basis functions for approximation.