Language modeling with stochastic automata

It is well known that language models are effective for increasing the accuracy of speech and handwriting recognizers, but large language models are often required to achieve low model perplexity (or entropy) and yet still have adequate language coverage. We study three efficient methods for stochastic language modeling in the context of the stochastic pattern recognition problem (variable-length Markov models, variable n-gram stochastic automata and refined probabilistic finite automata), and we give the results of a comparative performance analysis. In addition, we show that a method which combines two of these language modeling techniques yields an even better performance than the best of the single techniques tested.