An RNN-based noise estimation and likelihood compensation for noisy speech recognition

In this paper, a novel integration of RNN and PMC (parallel model combination) is presented for noisy speech recognition. It first employs an RNN to make the noise/speech discrimination. Then, by viewing the RNN outputs as the membership functions of noise and speech, an online noise tracking is performed for noise estimation. Also, a confidence measure is defined to represent the degree of the reliability of noise estimate and used to smooth the noise estimate across segments. The noise estimate is then used in PMC to adapt the hidden Markov models trained from clean speech. Finally, the RNN outputs are used to weight the likelihood scores of the PMC for softly reducing the influence of noise frame in the final decision. Experimental results showed that a significant improvement on recognition performance has been achieved under the non-stationary noise environment.