Adaptive Feedback Control of Linear Stochastic Systems

We consider adaptive control of linear stochastic systems, i.e., the control of unknown linear systems subject to stochastic disturbances whose spectra are also Unknown. We examine the basic convergence issues, including the convergence of adaptive controllers and parameter estimates as well as the convergence of input and output. Despite over a decade of effort, previous works in this area are very much fragmented. Relatively complete convergence results are available only for adaptive minimum variance control of unit delay systems. In this paper we propose the generalized certainty equivalence approach to stochastic adaptive control, where the estimates of disturbance innovations as well as parameter estimates are utilized. Based on this, the self-optimality of adaptive minimum variance controllers using an indirect approach and the stochastic gradient algorithm is established for general delay systems. Then we show that the self-optimality implies the self-tuning of adaptive controllers in general, by exhibiting the convergence of the parameter estimates to the null space of a certain covariance matrix and by characterizing the null space. The role of the system disturbance in providing an "internal excitation" is delineated. Finally we determine the exact order of external excitation required in order for the parameter estimates to converge to the true parameter.