Infinite-State Verification: From Transition Systems to Markov Chains

In this tutorial, we consider verification of Markov chains with infinite state spaces. We present a general framework which can handle probabilistic versions of several classical models such as Petri nets, lossy channel systems, push-down automata, and noisy Turing machines. First, we describe algorithms for verification of well quasiordered transition systems. These are transition systems which are monotonic w.r.t. a well quasi-ordering on the state space. Then, we extend the framework by introducing decisive Markov chains, a class of Markov chains which cover all the above mentioned models. We consider both safety and liveness problems for decisive Markov chains. Safety: What is the probability that a given set of states is eventually reached. Liveness: What is the probability that a given set of states is reached infinitely often. We will also consider limiting behaviors for infinite-state Markov chains. In order to do that, we consider a stronger condition than decisiveness, namely that of eagerness. Finally, for a subclass of the models, we describe algorithms to solve general versions of the problems in the context of simple stochastic games.