Risk-aware semi-Markov decision processes

In this work we construct a basic theory of risk-aware continuous-time Markov decision processes, and even more broadly, that of semi-Markov decision processes. Methods that account for the preferences of risk-aware agents have been introduced and studied in the context of discrete time problems, however, there has been virtually no such development for continuous-time models. We extend the literature of risk-aware optimization to semi-Markov control problems, and consider generic measures of risk with infinite-horizon discounted costs. We show that the optimization problem can be recast into a linear program using occupation measures, and can thus be solved using convex analytic methods. Our results extend the theory of risk-aware (discrete-time) Markov decision problems to the continuous-time setting, and allow optimization of e.g. risk-sensitive queuing systems.