From MDP to POMDP and Back: Safety and Compositionality

We propose a compositional framework for the stochastic safety of distributed Markov Decision Processes (MDPs) and Partially Observable Markov Decision Processes (POMDPs). We use MDP and POMDPs and their distributed versions as an appropriate modelling paradigm for computational ecosystems, understood in the context of distributed systems. We extend our work on stochastic safety from MDPs to POMDPs, and then to networked MDP/POMDPs. We propose a unifying mathematical framework for stochastic safety for MDPs, their partially observable version and their composition.