Convergence in unconstrained discrete-time differential dynamic programming

The conditions under which the original differential dynamic programming (DDP) algorithm can be expected to converge are investigated and modifications in the algorithm to improve convergence properties are proposed. Quadratic convergence of DDP requires that the stagewise Hessian matrices computed in the algorithm be positive definite. Three procedures for modifying the stagewise Hessian matrices to guarantee convergence for situations in which the stagewise Hessian matrices are not positive definite are proposed. One of the procedures (an active shift) converges quadratically. The computational requirements for DDP with and without the shift procedures are presented, and the impact of the ratio of state variable dimension to control variable dimension on the relative efficiency of the different shift procedures is discussed. Numerical results for a series of problems, including one with 16 state variables and 200 time steps, are presented. The most robust shift procedure is an adaptive shift that utilizes a constant shift followed by an active shift.<
>