Approximation of stochastic nonlinear closed-loop feedback control with application to miniature walking robots

We consider stochastic nonlinear time-variant systems with imperfect state information in the context of model predictive control. The optimal control performance can only be achieved by closed-loop feedback policies, which in fact anticipate future behavior. However, the computation of these policies is in general not tractable due to the presence of the dual effect, i.e., the control actions not only influence the state but also the uncertainty of its estimate. Thus, we propose an approximation to closed-loop control. We use a forward calculation approach, which is derived from an open-loop feedback control setup, but implements the fundamental property of closed-loop control that future measurement feedback is considered in the optimization. By using a finite set of representative measurements, the feedback behavior is anticipated only based on currently available information. The proposed optimization scheme is based on a continuation method, which implements an effective calculation to obtain a sequence of control inputs. The presented approach is evaluated by means of the control of a miniature walking robot.

I. Introduction

In Model Predictive Control (MPC), the main objective is to compute and apply control inputs such that the behavior of a system quantified by means of a cost function is optimized over a certain control horizon. Modelling errors or disturbances affecting the system can be considered in a stochastic fashion leading thereby to stochastic MPC (SMPC). The optimal solution, or strictly speaking the closed-loop optimal solution, is given by the Bellman equation, which unfortunately is only computable in few special cases, such as the linear quadratic Gaussian (LQG) control problem or the control of systems with a finite number of states and control inputs. This is mostly due to the curse of dimensionality and the fact that separation of estimation and control does not hold in more general system classes, as is the case with stochastic nonlinear systems [1], [2].