Integrating multiple knowledge sources for utterance verification in a large vocabulary speech understanding system

A set of techniques have been developed for utterance verification (UV) as applied to a large-vocabulary speech understanding system in a telecommunications application. The novel aspect of the UV procedures described in this paper is the integration of acoustic and linguistic knowledge sources for UV and speech recognition. The paper consists of two parts. The first part is an experimental study of the use of acoustic likelihood ratio-based UV techniques to the problem of rejecting portions of a hypothesized word string that, for whatever reason, may have been incorrectly decoded by a large-vocabulary speech recognizer. The second part of the paper describes a procedure for integrating the acoustic-level confidence measures with the statistical language model. UV results are reported in this paper for utterances collected from a highly unconstrained call routing task performed over the telephone network. These techniques are evaluated in terms of their ability to classify utterances into a set of 15 semantic actions corresponding to call types that are accepted by the application.