Solution of Hamilton Jacobi Bellman equations

We present a method for the numerical solution of the Hamilton Jacobi Bellman PDE that arises in an infinite time optimal control problem. The method can be of higher order to reduce "the curse of dimensionality". It proceeds in two stages. First the HJB PDE is solved in a neighborhood of the origin using the power series method of Al'brecht (1961). From a boundary point of this neighborhood, an extremal trajectory is computed backward in time using the Pontryagin maximum principle. Then ordinary differential equations are developed for the higher partial derivatives of the solution along the extremal. These are solved yielding a power series for the approximate solution in a neighborhood of the extremal. This is repeated for other extremals and these approximate solutions are fitted together by transferring them to a rectangular grid using splines.