Acoustic Word Embedding Model with Transformer Encoder and Multivariate Joint Loss

As a representation of speech information, acoustic word embedding can enable query-by-example keyword search with low-resource speech data. An acoustic word embedding model with Transformer encoder and multivariate joint loss is presented in this article. First of all, on the basis of BLSTM, the model can extract richer information by adding a Transformer encoder, which improves representation ability of the embedding vector. Then, the contrast loss is improved according to the multiple negative samples, and combined with the hinge loss with distance characteristics, the model generates a more compact vector representation and improves the identification of features. Finally, considering the unbalanced characteristics of samples, the anti-focal loss is introduced to form a multivariate joint loss together with the above two losses, so as to make the model performance better. The model has achieved certain results in word recognition tasks and has certain competitiveness compared with other acoustic word models.