Causal Recursive Parameter Estimation for Discrete-Time Hidden Bivariate Markov Chains

An algorithm for causal recursive parameter estimation of a discrete-time hidden bivariate Markov chain is developed. In this model, a discrete-time bivariate Markov chain is observed through a discrete-time memoryless channel. The algorithm relies on the EM-based recursive approach developed by Stiller and Radons for hidden Markov models. A distinct advantage of the discrete-time hidden bivariate Markov chain model is that the sojourn time distribution of its observable process in each state is phase-type rather than geometric as in the hidden Markov model. Phase-type distributions can approximate any desired sojourn time distribution. Particular phase-type distributions include mixtures and convolutions of geometric distributions. The parameter estimation algorithm requires causal recursive estimation of the relevant statistics in each EM step. These statistics include the number of jumps of the bivariate Markov chain from one state to another, including self transitions, in the given observation interval, and first- and second-order statistical averages of the observable process in each state when the memoryless channel is Gaussian. We develop the explicit recursions and demonstrate the performance of the algorithm in estimating the model's parameter and its sojourn time distribution in a numerical example.