Cart (Loading....) | Create Account
Close category search window
 

The Poisson equation for reversible Markov chains: Analysis and application to Markov chain samplers

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$31 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

2 Author(s)
Cogill, R. ; Dept. of Syst. & Inf. Eng., Univ. of Virginia, Charlottesville, VA, USA ; Vargo, E.

The steady-state behavior a finite-state Markov chain can be evaluated by solving a Poisson equation, a special case of the optimality equation arising in average cost Markov decision processes. In practice, solving the Poisson equation is typically no easier than evaluating steady state behavior directly using the invariant probability mass function of the Markov chain. However, it is known that approximate solutions to the Poisson equation can be used to produce bounds on steady state performance and to accelerate simulations for estimating steady state behavior. In this paper we study the special structure taken on by the Poisson equation when the associated Markov chain is reversible. In particular, we show that reversible Markov chains have a special form of the Poisson equation that admits a closed form solution. As an application of the reversible Poisson equation we consider the construction of control variates that can be used in Markov chain-based samplers. We show one class of control variates that are obtained by approximating the known solution to the reversible Poisson equation, and demonstrate the application of these control variates on an example involving the Metropolis-Hastings algorithm.

Published in:

Decision and Control (CDC), 2012 IEEE 51st Annual Conference on

Date of Conference:

10-13 Dec. 2012

Need Help?


IEEE Advancing Technology for Humanity About IEEE Xplore | Contact | Help | Terms of Use | Nondiscrimination Policy | Site Map | Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest professional association for the advancement of technology.
© Copyright 2014 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.