Uncertainty estimation in automatic pronunciation assessment with pseudo samples based on deep kernel learning

A reliable and accurate automatic pronunciation assessment is of great help for second language (L2) learners. Commonly, a Gaussian process (GP)-based model is utilized to output pronunciation scores and their uncertainty. However, the application of GPs is limited by computational intractability when data are sufficiently numerous. In this paper, we propose an improved GP-based model for uncertainty prediction in au-tomatic pronunciation assessment utilizing a small set of pseudo samples derived from the full training set. To further exploit more expressive information from data, we optimize the network based on deep kernel learning with deep features derived from an automatic speech recognition (ASR) system. Experimental results based on one spoken test show its superiority compared with the GP-based baselines with the full training set and standard kernels. With a small number of pseudo samples, which is only 25% compared to the full training set, we can match the full GP performance.