Newton’s Method and Differential Dynamic Programming for Unconstrained Nonlinear Dynamic Games

Dynamic games arise when multiple agents with differing objectives choose control inputs to a dynamic system. However, compared to single-agent control problems, the computational methods for dynamic games are relatively limited. Only very specialized dynamic games can be solved exactly, so approximation algorithms are required. In this paper, we show how to extend a recursive Newton algorithm and differential dynamic programming (DDP) to the case of full-information non-zero sum dynamic games. We show that the iterates of Newton's method and DDP are sufficiently close for DDP to inherit the quadratic convergence rate of Newton's method.