On Reachability of Markov Chains: A Long-Run Average Approach

We consider a Markov control model in discrete time with countable both state space and action space. Using the value function of a suitable long-run average reward problem, we study various reachability/controllability problems. First, we characterize the domain of attraction and escape set of the system, and a generalization called
$p$
-domain of attraction, using the aforementioned value function. Next, we solve the problem of maximizing the probability of reaching a set
$A$
while avoiding a set
$B$
. Finally, we consider a constrained version of previous problem, where we ask for the probability of reaching the set
$B$
to be bounded. In the finite case, we use linear programming formulations to solve these problems.