Optimal control computation by the Newton-Raphson method and the Riccati transformation

A computational procedure is described for finding extremal control policies for nonlinear processes. The control problem is formulated as the problem of Bolza in the classical calculus of variations. The method is iterative in nature and prescribes a successive linearization of the Euler-Lagrange equations to obtain the extremal control. The resulting sequence of linear problems is solved such that the successive solutions may, in some well-defined manner, converge to the extremal solution. The linear two-point boundary value problem is decoupled by means of a generalized Riccati transformation. First, the matrix Riccati equation is integrated backwards in time. Then the state equations can be integrated forwards in time. The resulting curves can then be used for the next iteration. It is pointed out that in some instances the method yields a linear feedback control law, which is optimal with respect to initial condition perturbations. Also, the method allows for a sufficieney check. At the termination of the iterations, Jacobi's condition and the strengthened Legendre condition can be checked to see whether or not the control obtained is indeed optimal in the sense of a weak local minimum. Numerical results are presented and a brief comparison is made between this method and the First and Second Variation methods.