Near-optimal stopping rules for two-time-scale Markovian systems

Many stochastic systems in various applications are large and complex. It is difficult to find closed-form optimal control policies. In this paper, a representative of such systems is considered, namely, an optimal stopping problem driven by a finite state Markov chain. Although the state space is finite, it is large containing many elements resulting in computational complexity. To overcome the difficulty, we divide the states into a number of groups so that the chain jumps frequently within each group and jumps over to other groups only occasionally. Based on this model, we use a singular perturbation approach to reduce the overall dimensionality and design near-optimal strategies for the original problem. Examples will be provided to illustrate the results.