Recursive and optimal multi-level algorithms for the identification of stochastic systems applications to bipeds control and identification

We present a generalisation of doubly stochastic processes intelligent agents processing based hidden Markov models to non linear stochastic identification and control of dynamic systems with state representation. This stochastic approach is applied to the control and identification of a humanoid robot. This stochastic generalisation has two aspects: First, the hidden Markov models application to control and identification of non linear systems is basically of global and off-line nature. The globality of HMM is in its off-line characteristic. Our approach is control and estimation oriented of hidden Markov based modeling. A new global control based on the optimisation of the auxiliary function used in the hidden Markov models strategy is proposed. The convergence to optimal control is analysed. The second part of this communication will consider the on-line stochastic control and identification of non linear dynamic systems. We develop the Ergodic Algorithm (EA). EA permits the identification and control of a large class of stochastic non linear systems. The convergence analysis is established using reference probability and martingales approaches by defining a new model, a new space, and a new law of probability. Under this new law, we show the convergence of the parameters to the truth and desired values. Then we show the absolute contiguity of the new law of probability with the initial one. A new control and performance index is also defined and the stability of this control is considered. We present in this article the case study of the control and stabilisation of bipedal robot walk. Simulations show the high efficiency of HMM and EA.