FORESEE: Prediction With Expansion–Compression Unscented Transform for Online Policy Optimization

We introduce a method for the nonlinear state prediction, called the expansion–compression (EC) unscented transform (UT), and use it to solve a class of online policy optimization problems. The proposed algorithm includes an expansion operation, which propagates a finite number of sigma points through a state-dependent distribution, and a compression operation based on moment matching, which keeps the number of sigma points constant across predictions over multiple time steps. The performance of the algorithm is empirically shown to be comparable to Monte Carlo (MC) but at a much lower computational cost. The state prediction is used in tandem with a proposed variant of constrained gradient descent (GD) for online update of policy parameters in a receding horizon fashion while accounting for state and control input constraints. The framework is implemented as a differentiable computational graph for policy training. We showcase our framework with simulation and experimental case studies. The former include the stabilization of a quadrotor as part of a benchmark comparison in safe-control-gym, and the optimization of the parameters of a control barrier function (CBF) controller. The latter includes the on-the-fly optimization of tracking-controller gains for a quadrotor under wind disturbances.