By Topic

Two-Time Scale Controlled Markov Chains: A Decomposition and Parallel Processing Approach

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$33 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

2 Author(s)
Haurie, A. ; Univ. de Geneve, Chene-Bougeries ; Moresino, F.

This correspondence deals with a class of ergodic control problems for systems described by Markov chains with strong and weak interactions. These systems are composed of a set of subchains that are weakly coupled. Using results already available in the literature one formulates a limit control problem the solution of which can be obtained via an associated nondifferentiable convex programming (NDCP) problem. The technique used to solve the NDCP problem is the Analytic Center Cutting Plane Method (ACCPM) which implements a dialogue between, on one hand, a master program computing the analytical center of a localization set containing the solution and, on the other hand, an oracle proposing cutting planes that reduce the size of the localization set at each main iteration. The interesting aspect of this implementation comes from two characteristics: (i) the oracle proposes cutting planes by solving reduced sized Markov Decision Problems (MDP) via a linear program (LP) or a policy iteration method; (ii) several cutting planes can be proposed simultaneously through a parallel implementation on processors. The correspondence concentrates on these two aspects and shows, on a large scale MDP obtained from the numerical approximation ldquoa la Kushner-Dupuisrdquo of a singularly perturbed hybrid stochastic control problem, the important computational speed-up obtained.

Published in:

Automatic Control, IEEE Transactions on  (Volume:52 ,  Issue: 12 )