LNACont: Language-Normalized Affine Coupling Layer with Contrastive Learning for Cross-Lingual Multi-Speaker Text-to-Speech

The current advancement in text-to-speech (TTS) has achieved a commendable level of reproducing human-like voices, including diverse speaking style such as multiple speaker identities and emotions. Due to the difficulty of obtaining bilingual speaker data, cross-lingual multi-speaker TTS is typically trained by combining monolingual data. At this time, because each speaker's linguistic features are biased, the pronunciation clarity and speech quality are degraded when synthesizing speech for foreign languages not spoken by a specific speaker in the data. To tackle this, our study proposes two key methodologies. Firstly, we introduce a language-independent learning method by utilizing the speaker-normalized affine coupling layer (SNAC). This approach aims to enhance the separation of language specific features during the training process. Secondly, we employ contrastive learning and intercross training techniques. This involves ensuring that the distance between language and speaker embeddings is maximized, while simultaneously force speaker embedding to learn text-independent information. This strategy contributes to the effective segregation of language and speaker information. Through these, we observed that both the accuracy and quality of the cross-lingual speech synthesis results were improved.