By Topic

Decentralized control of finite state Markov processes

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$33 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

2 Author(s)
Kai Hsu ; Scientific Systems, Inc., San Antonio, TX, USA ; Marcus, S.I.

We are concerned with the control of a particular class of dynamic systems-finite state Markov chains. The information pattern available is the so-called one step delay sharing information pattern. Using this information pattern, the dynamic programming algorithm can be explicitly carried out to obtain the optimal policy. The problems are discussed under three different cost criteria-finite horizon problem with expected total cost, infinite horizon problem with discounted cost, and infinite horizon problem with average expected cost. The solution of the problem is possible with the one step delay sharing information pattern because, as in the centralized control of Markov chains, a separation principle holds (this is not true for multiple step delay sharing). Hence, the decentralized problem can essentially be reduced to a (more complicated) centralized one: this reduction is carried out in detail. With some modifications, the "policy iteration algorithm" and "Sondik's algorithm" are readily applied to find the optimal policies for these problems.

Published in:

Automatic Control, IEEE Transactions on  (Volume:27 ,  Issue: 2 )