Koopman Operator Approach to Optimal Control Selection Under Uncertainty

Uncertainty propagation is an important step in the derivation of optimal control strategies for dynamic systems in the presence of state and parameter uncertainty. Many stochastic control formulations seek to optimize an expected value of a score or cost function, or otherwise enforce a probabilistic constraint through the use of an expected value. The time-varying state density needed for this expected value computation may be quantified through Monte Carlo methods or explicit approaches such as the Frobenius-Perron operator. This paper explores an alternative approach to the computation of the expected value by leveraging the adjoint relationship between the Frobenius-Perron operator and the Koopman operator. This relationship states that the expected value of a forward-propagated density with a score function is equivalent to the expected value of the initial density with the score function mapped back to the initial state space. By computing the expected value through use of the Koopman operator, several computational issues arising when using the Frobenius-Perron operator can be mitigated, while maintaining the benefits of an explicit method over Monte Carlo. A general stochastic optimal control problem is formulated wherein the expected value is computed between the initial state density and a score function using the Koopman operator. The computational benefits of computing the expected value through use of the Koopman operator are discussed, particularly with respect to numerical issues encountered using explicit forward propagation of the state probability density. A series of numerical examples highlight these computational advantages and illustrate the benefit of using this adjoint approach in solution to a range of optimal control problems in the presence of uncertainty.