Necessary and Sufficient Certificates for Almost Sure Reachability

We consider the almost sure reachability problem for discrete-time stochastic dynamical systems, which asks if a system reaches a given subset of its state space almost surely (i.e., with probability one). We show necessary and sufficient conditions for almost sure reachability under suitable regularity assumptions on the system. Our conditions are in the spirit of Lyapunov theory, which reduces the problem of checking a global stability property of a dynamical system to checking properties of appropriate certificates. As certificates, we use supermartingales, which are functions that do not increase in expectation, for estimating the likelihood of the system’s evolution, and use variant functions for measuring distances to the target set. Given candidate supermartingale and variant functions, our conditions provide locally checkable conditions for almost sure reachability. We also show a converse theorem, which shows how to construct a suitable supermartingale and variant if a system satisfies the almost sure reachability property.