Phonetically conditioned prosody generation for TTS: An unsupervised phonetic-to-prosodic mapping framework

We propose a framework for phonetically-conditioned prosody generation for TTS which has two phases: first, in an off-line phase, it learns the phonetic-to-prosodic mapping underlying a target prosody database in an unsupervised manner using a `segmental K-means' algorithm; secondly, for a given input text, it uses a segmental unit-selection framework to select appropriate prosodic-profiles from the learnt mapping, conditioned on the input phonetic sequence, which are then transferred onto a conventional TTS output unit-sequence. This framework is derived from a 2-stage unit-selection framework we had proposed recently, which employs `phonetic conditioning', wherein target prosody-profiles are identified from a given target prosody database, conditioned on their underlying phonetic content with varying degrees of matching and over variable length time-scales that are optimized to be as long as possible. In this paper, based on an equivalence result shown by us earlier in ultra-low bit-rate speech coding, we show how the unit-selection framework can be approximated by a segmental unit-selection framework, with advantages of longer phonetic conditioning and very low complexity. We characterize the performance of the proposed framework using various objective measures, compare it with the earlier unit-selection framework and show its practical viability.