Modeling speech variability with segmental HMMs

An important characteristic of segmental hidden Markov models (HMMs) is a distinction between extra-segmental variability across different examples of a sub-phonemic speech segment and intra-segmental variability within any one example. This paper considers the modeling of these two types of variability in some detail. Analysis of natural speech data has suggested that intra-segmental variability of mel cepstrum features is not well-approximated by single Gaussians. The model of this variability was improved by introducing a two-component Gaussian mixture, where the two components have the same mean but one has much smaller variance than the other. Initial recognition experiments using models with three segments per phone have shown the importance of accurately modeling intra-segmental variability, in order for the intra- and extra-segmental probabilities to balance correctly and hence for good recognition performance to be achieved.