Scheduled System Maintenance on May 29th, 2015:
IEEE Xplore will be upgraded between 11:00 AM and 10:00 PM EDT. During this time there may be intermittent impact on performance. For technical support, please contact us at We apologize for any inconvenience.
By Topic

A Bayesian network approach to control of networked Markov decision processes

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$31 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

3 Author(s)
Adlakha, S. ; Dept. of Electr. Eng., Stanford Univ., Stanford, CA ; Lall, S. ; Goldsmith, A.

We consider the problem of finding an optimal feedback controller for a networked Markov decision process. Specifically, we consider a network of interconnected subsystems, where each subsystem evolves as a Markov decision process (MDP). A subsystem is connected to its neighbors via links over which signals are delayed. We consider centralized control of such networked MDPs. The controller receives delayed state information from each of the subsystem, and it chooses control actions for all subsystems. Such networked MDPs can be represented as partially observed Markov decision processes (POMDPs). We model such a POMDP as a Bayesian network and show that an optimal controller requires only a finite history of past states and control actions. The result is based on the idea that given certain past states and actions, the current state of the networked MDP is independent of the earlier states and actions. This dependence on only the finite past states and actions makes the computation of controllers for networked MDPs tractable.

Published in:

Communication, Control, and Computing, 2008 46th Annual Allerton Conference on

Date of Conference:

23-26 Sept. 2008