Review of Stochastic Process

This chapter presents the basic concepts of stochastic process. The main focus is on discrete‐time and continuous‐time Markov processes with an application to power systems reliability. In engineering applications, a discrete‐time homogeneous Markov process can be represented by a state‐transition diagram, which shows all possible states as well as the transition probabilities. In a discrete‐time Markov process, a state can be classified in the state space using transition probability. For a finite‐state discrete‐time Markov process, the state space can be decomposed into disjoint sets of transient and recurrent states. It is also of interest to find the time that a process encounters a state for the first time, called first passage time. For finding the mean first passage time, some transition probabilities may need to be set to zero to create appropriate sets of transient and recurrent states.