Phoneme-Viseme Mapping for Sinhala Speaking Robot for Sri Lankan Healthcare Applications

Speech perception is considered entirely as an auditory process, but vision also has a significant influence on speech perception. In generating synthesized vocal systems, especially for robotic applications, inaccurate synchronization between voice and lip movements substantially decreases the speech understanding and the naturalness of face-to-face communication. Phoneme-viseme mapping is one of the most important approaches in the visual recognition of speech and visual speech synthesis applications. Although there are many phoneme-viseme mapping models for languages such as English, Indonesian, Arabic, German, no adequate phoneme-viseme mapping model is available for the Sinhala language. This research proposes a methodology for Sinhala static viseme classification and establishes a phoneme-viseme mapping model for the Sinhala language. The Sinhala language is a low-resource language that belongs to the Indo-European sub-family, it has some similarities to the languages like Hindi, Marathi, and Bengali. The traditional Sinhala phonetic alphabet consists of 40 phonemes including 14 vowels and 26 consonants. This paper outlines the analysis of geometrical lip movements and features of the speakers pronouncing Sinhala word sequences which have been recorded in optimal conditions. Viseme classes are obtained through a static viseme approach where K-means clustering techniques and Sinhala linguistic features are considered. The proposed model was validated through a subjective analysis method and this is expected to grow into a reference model for future research attempts, as well as for developing an instructional robotic face that will form the visual interface for Sinhala-speaking healthcare seekers.