Unsupervised Spoken Term Discovery Using wav2vec 2.0

Unsupervised spoken term discovery is the task of finding recurring word-like patterns from raw audio without any manual transcription. Several approaches have been investigated, but the matching between automatically found fragments and actual words is still shallow. Recently, a self-supervised learning method wav2vec 2.0 has been proposed, and it is demonstrating outstanding performance in pre-training acoustic models for speech recognition. During the training, wav2vec 2.0 applies quantization to a latent representation of the input acoustic features. As a by-product, a discrete code sequence is obtained. In this work, we propose to use the code sequence for unsupervised term discovery. The temporal resolution of the code sequence is fine-grained, and it is closer to a phone sequence rather than a word sequence. To obtain larger units, we apply the ES-KMeans method to the code and feature sequences obtained by wav2vec 2.0. In addition, we iteratively optimize wav2vec 2.0 and ES- KMeans for further improvement. Experimental results using the Zero Resource Speech Challenge 2020's data show the proposed method outperforms existing methods on average while the results vary on languages.