Optimal Control of State-Affine Dynamical Systems

Optimal control of state-affine systems with finite or infinite dimensions is considered. The control performance is measured by a cost functional with state-affine Lagrangian and terminal cost. Relying upon such affine structure, a simple proof of Pontryagin's maximum principle as a necessary condition for optimality is presented. This principle requires any optimal control to resolve a certain two-point boundary value problem. As the main contribution of this paper, an iterative algorithm is proposed that converges to the solution of this boundary value problem. This solution is regarded then as a candidate optimal control. Several applications are outlined for the optimal control problem of this paper, including: optimal control of unobserved stochastic systems (continuous-time Markov chain and diffusion process), convection-diffusion partial differential equations, and Lyapunov matrix differential equations.