Prosodic Chinese Sign Language Synthesis Driven by Speech

Making virtual human's actions more natural and more approaching to reality is helpful for enhancing intelligibility of synthesized Chinese sign language (CSL). Usually action with prosody looks more vivid and prosody means that same word has discriminating style according to different context. The problem here is how to find and indicate the context. As formal description for CSL, CSLML provides a set of top level tags for recording context. This paper provides a new method for finding prosody information of context, which is utilizing speech sounds. Hidden Markov model (HMM) is trained for finding different prosody patterns, such as word stressing, sentence modal, and emotions etc. And parameters of HMM are mapped to value of tags in CSLML. The prosody tags are then transferred to animation parameter to synthesize prosodic Chinese Signed Language expression. Experiments in this paper show that synthesized animation driven by prosody parameters extracted with HMM is more realistic.