Combining Numerical Linear Algebra with Simulation to Compute Stationary Distributions

This paper introduces the first fully integrated algorithm for combining simulation with numerical linear algebra, as a means of computing stationary distributions for Markov chains and Markov jump processes. We use linear algebra to analyze the “center” of the state space, while simulation is used to estimate contributions to the steady-state from path excursions outside the “center”. The method yields consistent estimators for stationary expectations, and can be viewed as an application of the variance reduction technique known as conditional Monte Carlo.