Evaluation of a Korean Lip-sync system for an android robot

Lip-syncing of android robots resembling people is essential to accurately convey their intentions to humans. In this paper, we develop a system of Korean lip-syncing, with the assumption that people can guess a word or phrase from watching a lip-syncing robot without sound. The mouth shape for 10 single vowels was generated based on a Korean single vowels triangle chart. Robots can lip-sync in real time a variety of words and sentences using 10 mouth shapes. We performed experiments recording a mouth robot and an announcer reading text. We conducted a survey to assess humans guessing the representations of a female announcer and of a robot to compare the percent of correct answers in each case. Additionally, we also conducted a survey of robot mouth shapes and lip-sync timing to assess the reaction of subjects on 5-Likert scales. Results indicate that the percent of correct guesses from the mouth shape of the robot was one third of that from the human announcer. Subjects assessed the mouth shape and lip-sync timing of the robot as being somewhat unnatural. We expect that android robot lip-syncing currently uses mouth shapes that are perceived as lying in the uncanny valley when subjects try to interpret them. Thus, we will present a more natural mouth shape, add mouth shapes for diphthongs, and develop a mouth shape that varies with voice volume, improving the rate of lip-sync recognition.