Investigation of Text-to-Speech-based Synthetic Parallel Data for Sequence-to-Sequence Non-Parallel Voice Conversion

Sequence-to-sequence (seq2seq) voice conversion (VC) can achieve high-quality VC performance with a parallel dataset, but it is still challenging when the parallel dataset is not available. One way to overcome this issue is using synthetic parallel data (SPD) produced by text-to-speech (TTS) models trained with source and target speakers' voices in an available VC dataset. In this paper, we conduct a systematic study on the effects of SPD on seq2seq VC performance. Some factors, such as a comparison of SPD by source or target speaker, the effects of SPD in a semiparallel setting including a parallel subset, and the usage of SPD with external text data are investigated. The results show the effects of SPD depend on TTS performance and VC training datasize; i.e., 1) when the datasize is small, causing SPD quality degradation as the resulting TTS performance is limited, the training pair containing source SPD and target natural speech tends to yield slightly better VC results than that containing source natural speech and target SPD, 2) although SPD makes it possible to use a nonparallel dataset, using parallel subset is still effective, and 3) SPD with external text data as data augmentation can improve parallel seq2seq VC performance.