Triplet Confidence for Robust Out-of-vocabulary Keyword Spotting

Keyword Spotting (KWS) is a task that detects predefined keywords in a stream of audio. Although state-of-the-art deep neural networks perform well on KWS, they are not robust against out-of-vocabulary (OOV) samples because of their over-reliance on labeled data and the great imbalance between in-vocabulary (IV) and OOV data resulting from the infinity of OOV data. Besides, some end-to-end(E2E) KWS try to treat it as a multi-classification task, rejecting most OOV samples through posterior processing, but they cannot ensure the classifier can always be sure that its judgment is correct and keywords are usually too short to extract many representative features. To address these issues, we introduce a KWS model that can keep robustness on OOV samples by learning confidence estimates of the model. Confidence estimation is output by the self-attentional confidence branch, which can focus on single keyword in context. And we propose a loss function, named Triplet Correction Loss, which learning confidence estimation to improve the reliability of the model without relying on labeled data. Compared with the state-of-the-art methods, our proposed network increases 1.80%(V2-25) and 7.63%(Librispeech) accuracy on OOV samples, while keeping the high accuracy on IV dataset. We also provide ablation study to prove that our method is effective.