By Topic

An optimization-oriented approach to the adaptive control of Markov chains

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$31 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

2 Author(s)
Milito, R.A. ; University of Illinois, Urbana, Illinois ; Cruz, J.B.

We consider the control of a dynamic system modeled as a Markov chain. The transition probability matrix of the Markov chain depends on the control u and also on an unknown parameter α. The unknown parameter belongs to a given finite set A. The long run average cost depends on the control policy and the unknown parameter. Thus a direct approach to the optimization of the performance is not feasible. A common procedure calls for an on-line estimation of the unknown parameter and the minimization of the cost functional using the estimate in lieu of the true parameter. It is well-known that this "certainty equivalence" (CE) solution may fail to yield the optimal performance. This motivates the presentation of a new optimiza tion-oriented approach to adaptive control. We consider a composite functional which simultaneously take care of theestimation and control needs. The global minimum of this composite functional coincides with the minimum of the original cost functional. Thus its joint minimization with respect to control and parameter estimates would yield the optimal control policy This joint minimization is not feasible, but it suggests an algorithm that asymptotically achieves the desired goal. Due to space constraints we omit a review of the literature as well as the proofs of our claims. They will be presented elsewhere.

Published in:

Decision and Control, 1984. The 23rd IEEE Conference on

Date of Conference:

12-14 Dec. 1984