New Insights into Turbo-Decoding-Based AVSR with Dynamic StreamWeights

Automatic speech recognition (ASR) is an important component of many technical systems which enables seamless and intuitive human-machine interaction. However, the accuracy of ASR drops considerably when speech signals are degraded, e.g., by noise or reverberation. The introduction of a second signal stream which is not affected by the signal degradations, e.g., a video stream, helps to improve ASR such that it becomes more robust against these degradations. To incorporate information from different signal streams to an ASR system, different decoding strategies have been proposed, where especially turbo-decoding (TD) based ASR with optimally weighted streams has yielded high recognition rates even in severe acoustical conditions. The development of an algorithm for the estimation of these optimal dynamic stream weights (DSW) opened up new research questions, e.g., w.r.t. to the characteristics of DSWs and the evaluation of the recognition accuracy over the iterations of TD, which are covered in this contribution.