NAR-SREC: Nonautoregressive End-to-End Speech Recognition With Error Correction Decoder

With the popularity of mobile smart devices, speech recognition utilizes the speech signals received by sensors to quickly find the most likely text or command through nonautoregressive (NAR) inference, providing a solid foundation for human-computer interaction. However, NAR inference ignores the dependencies between words, which significantly degrades recognition accuracy. To alleviate this problem, we propose a novel NAR end-to-end (E2E) speech recognition with error correction decoder (NAR-SREC), which improves recognition accuracy and inference speed by closely collaborating with three core components. Specifically, we first design a new context-association CTC (CACTC) to incorporate explicit linguistic information into the output condition term for NAR inference, which enhances interword relationship learning. Then, we propose a new BERT-augmented module to simultaneously model the target sequence and source sequence, where the former provides linguistic information to the CACTC and the latter performs error probability detection. Finally, we integrate an NAR error correction (NAR-EC) decoder for fast EC, which focuses on correcting incorrect tokens to further reduce the insertion or substitution errors caused by the insufficient interword relationship. Extensive experiments on Aishell-1 and Librispeech show that NAR-SREC significantly improves the recognition accuracy compared with other advanced NAR methods, achieving a better tradeoff between speed and performance.