Approximation of stochastic nonlinear closed-loop feedback control with application to miniature walking robots

We consider stochastic nonlinear time-variant systems with imperfect state information in the context of model predictive control. The optimal control performance can only be achieved by closed-loop feedback policies, which in fact anticipate future behavior. However, the computation of these policies is in general not tractable due to the presence of the dual effect, i.e., the control actions not only influence the state but also the uncertainty of its estimate. Thus, we propose an approximation to closed-loop control. We use a forward calculation approach, which is derived from an open-loop feedback control setup, but implements the fundamental property of closed-loop control that future measurement feedback is considered in the optimization. By using a finite set of representative measurements, the feedback behavior is anticipated only based on currently available information. The proposed optimization scheme is based on a continuation method, which implements an effective calculation to obtain a sequence of control inputs. The presented approach is evaluated by means of the control of a miniature walking robot.