MetricCycleGAN-VC: Forcing CycleGAN-Based Voice Conversion Systems to Associate With Objective Quality Metrics

In CycleGAN-based voice conversion (VC) systems, such as MaskCycleGAN-VC, similar to many other GAN-based VC systems, adversarial losses are not explicitly designed to optimize VC evaluation metrics. Consequently, the generator may not consistently improve the metric scores related to VC assessments, such as the Mel-Cepstral Distance (MCD). To address this, we propose MetricCycleGAN-VC, which optimizes discriminators based on one or more VC evaluation metrics in a black-box strategy. Our approach covers various metrics related to VC quality triple aspects: speaker similarity, speech naturalness, and intelligibility. We Examine different well-known speech-related metrics such as MCD, Short-Time Objective of Intelligibility (STOI), neural tools such as Non-Intrusive objective Speech Quality Assessment (NISQA), and a newly proposed metric called Speaker Embedding Cosine Similarity (SECS). We also leverage full-reference image quality metrics like Feature-based similarity index Measure (FSIM) and combine all these intrusive, and non-intrusive metrics Within a unified framework for non-parallel voice conversion. While our method allows the utilization of all existing or newly developed quality metrics, this will be done without compromising their structural integrity or requiring knowledge of their formulation. Experimental results demonstrate improved desired metrics in both inter- and intra-gender conversions, showcasing the superiority of our approach over MaskCycleGAN-VC a typical baseline and many state-of-the-art VC systems. A key strength of our proposed method is its adaptability to evolving VC quality metrics, promising even better results as more precise metrics are introduced and implemented.