Controlled Markov chains with risk-sensitive exponential average cost criterion

In this paper we are concerned with the exponential risk-sensitive version of the standard average cost criterion for controlled Markov chains (CMC). Our presentation is mathematically rigorous, and our proof techniques are self-contained and perhaps somewhat intuitive. Furthermore, we extend some previous results to the countable state space case. In addition, we consider optimization within the general set of randomized policies, and not only within the restricted class of Markovian deterministic policies. We model risk sensitivity as being given by an exponential disutility function U/sub /spl gamma//(x)=(sgn/spl gamma/)e/sup /spl gamma/x/, where /spl gamma/ is the constant risk-sensitivity coefficient. After basic definitions and notation, the paper presents and briefly analyzes alternative definitions of the exponential average cost criterion (EAC). Howard and Matheson's definition of EAC (1972) is discussed in detail. Finally we show that, similarly to the risk-neutral case, the optimal EAC satisfies an optimality equation.