TSCL: Two-Stage Self-Supervised Contrastive Learning for Spoken Language Identification

Spoken language identification (SLI) is an important field of acoustic classification. As the front-end technology of many multilingual systems, it is widely applied. However, the field of language identification based on acoustic features lacks adaptive methods akin to image augmentation and feature distillation techniques prevalent in the image domain for enhancing the category-specificity of the obtained features. To address this, we propose innovations in two aspects. On the one hand, considering the characteristics of acoustic features, we propose Phoneme-Level Unit Replacement (UPLR) and Temporal Co-location Merging (TCM) for data augmentation. On the other hand, we subject both the augmented data and the original data to a Contrastive-aware Dual Attention Feature Distillation (CDAFD) framework for feature enhancement, yielding a reusable feature enhancement apparatus. This two-stage feature enhancer comprises two phases. In the first stage, we employ our proposed Swin Transformer model, where calculating the InfoNCE loss for input sample pairs belonging to the same category and perform backpropagation to update the structure of this Swin Transformer model. Within the traditional Swin Transformer architecture, we integrate three novel structures: Gated Cross-stage Propagation (GCSP), Modality-aware Cross-Attention (MACA), and Local Feature Retention Pathway (LFRP). In the second stage, we construct a Self-Attention layer and a Down-Projection layer to perform feature enhancement, where updating the parameters of these aforementioned structures by calculating the Category Aggregation Loss (CAL). Ultimately, we obtain a feature extractor. We validated the effectiveness of the proposed methods through experiments.