Phoneme-Level Duration Controllable Neural Text-to-Speech With Phoneme Embedding Skip Connection and Modified Gaussian Duration Modeling

This paper proposes a simple but practically important and effective approach to improve phoneme duration expansion and contraction control in neural text-to-speech (TTS) systems for modifying the speaking rate of synthesized speech. The use of simple uniform expansion and contraction is sometimes unnatural because the expansion and compression rates of actual human speech are different for each phoneme in fast and slow conditions. To address this problem, this paper introduces a phoneme embedding skip connection technique (PESC), in which phoneme-level embeddings are added with the encoder output features in neural TTS acoustic models by a simple skip connection, and phoneme-level components can then be enhanced. By enhancing the phoneme-level components before the length regulator, PESC can improve the synthesized speech quality for phoneme duration expansion and contraction conditions. Additionally, this paper proposes modified Gaussian duration modeling (GDM), which predicts not only mean values but also variance values of phoneme durations, for neural TTS acoustic models. Combined with PESC, the modified GDM can further improve the synthesized speech quality for the duration expansion condition. Objective and subjective experiments are conducted for both whole sentence duration control and partial utterance duration control conditions with both Japanese and English neural TTS systems. The results of the experiments demonstrate that PESC without GDM outperforms conventional WSOLA, feature interpolation, and simple expansion and contraction methods for the duration contraction condition, and that PESC with GDM outperforms the conventional methods for the duration expansion condition.