Retrieval-oriented E2E ASR Modeling for Improved Query-by-example Spoken Term Detection

Query-by-example Spoken Term Detection (STD) systems can make effective use of automatic speech recognizer (ASR), especially in situations with high recognition accuracy. However, the conventional DNN-HMM ASR has a problem of low recognition accuracy for out-of-vocabulary (OOV) words. This problem has a serious impact on the performance of STD-based speech retrieval and can lead to false retrieval. In a recent study, End-to-end (E2E) ASR has achieved comparable performance compared to DNN-HMM ASR. Since E2E ASR can learn direct speech-to-character mappings without knowledge of word or phonetic pronunciation, it can reduce the impact of the OOV problem. In this paper, we propose STD method using an ASR system adapted to spoken word retrieval and acoustic similarity at the sub-phone level. Our proposed method employs E2E ASR modeling with Kana as the output unit, which can be easily converted to phoneme sequence. To improve the recognition and retrieval accuracy of words that are likely to be searched for, we propose E2E modeling that adds optional tag information to the output unit. Experiments using the NTCIR-12 SpokenQuery&Doc-2 task show that the E2E ASR-based STD method significantly improves the retrieval performance over the STD method using DNN-HMM ASR. This is due to the fact that the proposed E2E ASR was able to reduce the OOV problem for spoken documents and spoken queries.