Non-Intrusive Signal Analysis for Room Adaptation of ASR Models

We present a new deep-learning-based non-intrusive signal assessment method (NISA+) that performs a joint estimation of a large set of speech signal parameters, including those related to reverberation (C
50
, DRR, reflection coefficient and room volume), background noise (SNR), perceptual speech quality (PESQ), speech intelligibility (ESTOI), voice activity detection, and speech coding (codec presence and bitrate). We show that neural embedding based combination of spectral features with an LSTM and modulation features with a convolution neural network enable NISA+ to achieve state of the art performance. Particularly, for non-intrusive PESQ and C
50
estimation, we show around 15% relative reduction in estimation error compared to our previous best results. We also show that NISA+ can be used to perform targeted data augmentation for generating training data for ASR that matches the signal characteristics extracted from a small sample of data recorded in a target room acoustic environment. We show that a 9.6% word error rate reduction can be achieved relative to an ASR model trained with random augmentation.