Joint distributional modeling with cross-correlation based features

In maximum likelihood based speech recognition systems, it is important to accurately estimate the joint distribution of feature vectors given a particular acoustic model. We propose that by modeling the joint distribution of time localized feature vectors and statistics relating those time localized feature vectors to the relevant acoustic context, we can estimate information contained in the feature vector joint distribution without the accompanying theoretical or computational difficulties. We introduce the modcrossgram (MCG), a computational way of estimating short time spectro temporal correlation based statistics that are informative about the feature vector joint distribution. Using the standard hybrid ANN/HMM architecture, we compare a MCG based speech recognition system with a more traditional one on an isolated word speech database. We show that, in the presence of noise, the MCG based system achieves a significant reduction in word error rate over the standard system.