Controllable voice conversion based on quantization of voice factor scores

Various voice conversion models using deep learning have been proposed. However, most focus on speaker identity con-version from one person to another. To the best of our knowledge, there is no model that controls the factors that determine voice tone (e.g. husky and nasal) and transforms it. We propose voice factor quantized splitternetVC (VFQ-SplitterNetVC), a voice conversion model that can control voice tone by learning factors that determine voice tone, such as the degree of huskiness, and their strength and weakness, based on SplitterNetVC and vector quantized-variational auto encoder (VQ- VAE). We evaluated the naturalness of the converted voice, speaker similarity, and voice tone control performance of the proposed model in subjective experiments. We found that VFQ-SplitterNetVC showed high speaker similarity and voice tone control performance.