Use of word level side information to improve speech recognition

Word level information obtained from the output of a speech recognizer has been used in the past to extract confidence features for the hypothesized words. This work describes a post-recognition process which treats these word-level features as independent knowledge sources and combines them in one log linear model for the posterior probability of a word sequence. This model is used for rescoring the hypotheses. The parameters of the model are optimized using a discriminative model combination approach, where a simplex optimization method, known as amoeba search, is used to minimize the non-smooth function of empirical error rate on training data. The method is evaluated on the SWITCHBOARD database. After training 20 new parameters, we obtain a significant word error rate reduction over the baseline system. A correlation measure between features and word accuracy is defined to help analyze and explain the results.