A new approach to stochastic adaptive control

The principal techniques used up to now for the analysis of stochastic adaptive control systems have been 1) super-martingale (often called stochastic Lyapunov) methods and 2) methods relying upon the strong consistency of some parameter estimation scheme. Optimal stochastic control and filtering methods have also been employed. Although there have been some successes, the extension of these techniques to a broader class of adaptive control problems, including the case of time-varying parameters, has been difficult. In this paper a new approach is adopted: if an underlying Markovian state-space system for the controlled process is available, and if this process possesses stationary transition probabilities, then the powerful ergodic theory of Markov processes may be applied. Subject to technical conditions, such as stability, one may deduce 1) the existence of an invariant measure for the process and 2) the convergence almost surely of the sample averages of a function of the state process (and of its expectation) to its conditional expectation. The technique is illustrated by an application to a previously unsolved problem involving a linear system with unbounded random time-varying parameters.