Singularly perturbed Markov chains. II. Applications to controlled dynamic systems and Markov decision processes

For pt.I see ibid., p.1103-8 (1997). Markov decision process with weak and strong interactions, such that the states of the process can be divided into several groups such that transitions among the states within each group occur much more frequently than the transitions among the states belonging to different groups, are considered. By treating the states in each group as a single state, one derives a limit problem for discounted cost criteria. Given an optimal solution of the limit problem, one constructs a solution for the original problem which is asymptotically optimal. The paper discusses applications to Markov decision processes and nearly optimal controls of stochastic dynamic systems. Proofs are omitted.