By Topic

Controlled Markov chains with risk-sensitive average cost criterion: the non-irreducible case

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$31 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

2 Author(s)
Brau-Rojas, A. ; Departamento de Matematicas, Sonora Univ., Mexico ; Fernandez-Gaucherand, E.

We study discrete controlled Markov chains with finite state and action spaces. The performance of control policies is measured by a risk-sensitive average cost, the exponential average cost (EAC), which models risk-sensitivity by means of an exponential (dis)utility function. The main result is the characterization of the EAC corresponding to an arbitrary stationary deterministic policy in terms of the spectral radii of suitable irreducible matrices. This result generalizes a well known theorem of Howard and Matheson (1972) that deals with the particular case in which the transition probability matrix induced by the policy is primitive. It is shown that, when a stationary deterministic policy determines only one class of recurrent states, the corresponding EAC converges to the risk-null average cost as the risk-sensitivity coefficient goes to zero. However, it is also shown that for large risk-sensitivity, fundamental differences arise between both models. A proof of the existence of solutions to the associated optimality equation, under a simultaneous Doeblin condition and for small enough risk-sensitivity coefficient, is given. Our proof relies on the Perron-Frobenius theory of non-negative matrices. An example that shows the impact of risk-sensitivity on the Hernandez-Hernandez condition for the existence of solutions to an optimality inequality is constructed

Published in:

Decision and Control, 2001. Proceedings of the 40th IEEE Conference on  (Volume:3 )

Date of Conference:

2001