Carnival—Combining Speech Technology and Computer Animation

Speech is powerful information technology and the basis of human interaction. By emitting streams of buzzing, popping, and hissing noises from our mouths, we transmit thoughts, intentions, and knowledge of the world from one mind to another. We're accustomed to thinking of speech as an acoustic, auditory phenomenon. However, speech is also visible. Although the primary function of speech is to manipulate air in the vocal tract to produce sound, this action has an ancillary effect of changing the face's appearance. In particular, the action of the lips and jaw during speech causes constant deformation of the facial surface, generating a robust visual signal highly correlated with the acoustic one1-that is, visual speech.