Comparison of the order reducing (ORED) and fast incremental training (FIT) algorithms for training high order hidden Markov models

Du Preez (1997) detailed the ORED and FIT algorithms which are both applicable to the training of high order hidden Markov models (HMM). Due to the presence of local optima, the training algorithms are not guaranteed to converge to the same result. In this paper we use simulations as well as experiments on speech to investigate some differences between them. We show that the FIT algorithm requires a fraction of the computational requirements, while simultaneously providing better accuracy and generalisation compared to the ORED approach. The experiments indicate that the FIT algorithm provides a practical approach to training high order HMMs in circumstances which might ordinarily be considered as unfeasible.