Controlled Markov chains with risk-sensitive criteria: some (counter) examples

The paper is concerned with risk-sensitive versions of the standard average cost and discounted cost criteria for controlled Markov chains. Risk-sensitivity is modelled by means of the family of exponential disutility functions. First, we study the risk-sensitive average cost corresponding to a fixed stationary deterministic policy, for finite state space models. We examine some (counter) examples which illustrate how the behavior of the risk-sensitive model departs from that of the risk-null one in the non-irreducible case. Finally, we present an example of a controlled Markov chain with infinite state space which, as opposed to Jaquette's (1976) result for finite models, does not have ultimately stationary optimal policies with respect to the risk-sensitive (exponential) discounted cost. The controlled Markov chain in that example satisfies a simultaneous Doeblin condition, an assumption that enables the vanishing discount approach in the risk-null models. Thus, the example provides more evidence to think that obtaining the mentioned approach for the risk-sensitive models might be even harder than what Jaquette's result had already indicated.