By Topic

Mean-Variance Criteria for Finite Continuous-Time Markov Decision Processes

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$31 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

2 Author(s)
Xianping Guo ; Sch. of Math. & Comput. Sci., Zhongshan Univ., Guangzhou, China ; Xinyuan Song

This technical note deals with the mean variance problem (known as the average variance (AV) minimization problem) for finite continuous time Markov decision processes. We first introduce a so called G-condition which is weaker than the well known ergodicity and unichain conditions and sufficient for the finiteness of the AV of a policy. Also, we present an example of a policy having infinite AV when the G-condition is not satisfied. Under the G-condition we prove that the AV criterion can be transformed into an equivalent mean (or expected) average criterion by using a martingale technique and an observation from the canonical form of a transition rate matrix, and thus the existence and calculation of an AV minimal policy over a class of mean optimal policies are obtained by a policy iteration algorithm in an finite number of iterations. As byproduct, we obtain some interesting new results about the mean average optimality.

Published in:

Automatic Control, IEEE Transactions on  (Volume:54 ,  Issue: 9 )