Predicting the Viterbi Score Distribution for a Hidden Markov Model and Application to Speech Recognition

Hidden Markov models are used in many important applications such as speech recognition and handwriting recognition. Finding an effective HMM to fit the data is important for successful operation. Typically, the Baum-Welch algorithm is used to train an HMM, and seeks to maximize the likelihood probability of the model, which is closely related to the Viterbi score. However, random initialization causes the final model quality to vary due to locally optimum solutions. Conventionally, in speech recognition systems, models are selected from a collection of already trained models using some performance criterion. In this paper, we investigate an alternative method of selecting models using the Viterbi score distribution. A method to determine the Viterbi score distribution is described based on Viterbi score variation with respect to the number of states (N) in the model. Our tests show that the distribution is approximately Gaussian when the number of states is greater than 3. The paper also investigates the relationship between performance and the Viterbi score's percentile value and discusses several interesting implications for Baum-Welch training