MISGAN:Mutual Information-Based Unsupervised with Star-GANv2 Voice Conversion

Voice conversion (VC) is a technique that transforms the speech identity of a speaker to that of another while preserving the linguistic content. This technology has various applications such as dubbing for movies, cross-lingual conversion for language learning and speech assistance. However, many VC methods rely on parallel utterances to generate high-quality output. This requirement severely limits the practicality of this technology.We propose an unsupervised many-to-many VC model called MISGAN based on mutual information (MI). By combining adversarial loss and MI minimization loss, our model exhibits superior performance compared to previous VC models and can be employed for a range of voice conversion tasks, including arbitrary many-to-many, cross-lingual, and cross-gender voice conversion. We combine the spectral entropy (SE) of the speech signal with Mel frequency cepstral coefficients (MFCC) and use MI to reduce the correlation between different features, thereby improving the learning efficiency of the model. In addition, our model is fully convolutional and uses a faster and more stable WORLD vocoder for real-time voice conversion.Objective evaluation indicates that our proposed method improves speech quality both globally and locally. In addition, subjective evaluation indicates that our model is superior to Star-GANv2 in naturalness and speaker similarity.