Data Augmentation Based Unsupervised Pre-Training for Low-resource Speech Recognition

This paper proposes SpecWav2vec-F, a novel model built upon the Wav2vec 2.0 baseline. The model demonstrates enhanced effectiveness in low-resource speech recognition tasks by preserving relationships between different time steps in the latent speech space. It further improves performance by leveraging an augmented dataset. SpecWav2vec-F employs a new SpecAugment-based data augmentation method and combines a factorized time-delay neural network with the feature extractor of Wav2vec 2.0, enhancing robustness for better adaptation to scenarios with limited speech training data. In low-resource speech recognition tasks for English, Chinese, and Kazakh, the SpecWav2vec-F model achieves lower average word error rates compared to the baseline. Experiments on the KSC corpus with three supervised models—DNNHMM, E2E-LSTM, and E2E-Transformer—show that SpecWav2vec-F outperforms, with word error rates of 6.3% and 5.1% on the validation and test sets, respectively. These rates are 37% and 41.4% lower than the E2E-Transformer model and 4.5% and 5.6% lower than the baseline model on the same sets.