Acoustic and language modeling of human and nonhuman noises for human-to-human spontaneous speech recognition

Several improvements of our speech-to-speech translation system JANUS on spontaneous human-to-human dialogs are presented. Common phenomena in spontaneous speech are described, followed by a classification of different types of noise. To handle the variety of spontaneous effects in human-to-human dialogs, special noise models are introduced representing both human and nonhuman noise, as well as word fragments. It is shown that both the acoustic and the language modeling of the noise increase the recognition performance significantly. In the experiments, a clustering of the noise classes is performed and the resulting cluster variants are compared, thus allowing one to determine the best tradeoff between the sensitivity and trainability of the models.