Advanced language model fusion method for encoder-decoder model in Japanese speech recognition

We apply an advanced language model fusion method to the encoder-decoder models of a Japanese automatic speech recognition (ASR) system to reduce character error rates (CERs) in cross-domain scenarios. Our method, which uses a Density Ratio approach based on Bayes' theorem, is an extension of the Shallow Fusion method widely used for integrating the scores of speech recognition and language models. Since Japanese has a much larger number of characters than alphabet languages, and since there can be multiple possible meanings and pronunciations of the same kanji character, it is unclear whether the linguistic information inside a character-based end-to-end speech recognition is approximated by the language model. We conducted experiments using two types of encoder-decoder models, an RNN model and a Transformer model, and evaluated our results by calculating the cross-domain CERs of the ASR ‘ s text output when using the Japanese Academic Presentation Speech (APS) corpus and the Japanese Simulated Presentation Speech (SPS) corpus. When using the RNN model, our proposed method achieved a 1.2 % lower CER in comparison to the Shallow Fusion method. When using the Transformer model, our method also outperformed the Shallow Fusion method, achieving a 0.7% lower CER.