Multi-Encoder Sequential Attention Network for Context-Aware Speech Recognition in Japanese Dialog Conversation

Japanese automatic speech recognition (ASR) in conversation is considered challenging and highly ambiguous because the subjects are usually omitted and the number of homonyms is quite large. Since the speaker's utterance alone may not provide enough information, the context of the previous utterance from the dialog partner might help. A limited number of studies have addressed the idea of incorporating dialog-context information in end-to-end ASR tasks. However, these have mainly focused on the English language. Furthermore, although such models exploit dialog context history, the decoder handles the current input and the context simultaneously, ignoring the sequence of a conversation in the process. In this study, we propose a Japanese ASR that considers the dialog partner's conversation context in a sequential manner. Specifically, we use a multi-encoder sequential attention network and investigate several possible architectures. The experimental results reveal that context information helps to improve recognition performance over standard ASR and systems that process the context at the same time.