Bilabial Consonants Recognition in CV Persian Syllable Based on Computer Vision

According to previous researches, Persian consonants have been divided into seven categories based on viseme. It led to several consonants being placed in one category. Detecting between consonants in one category is so hard because the spots for the production of these consonants are the same. The forms of lips do not change at the time of production; these consonants are hardly distinguishable. The major challenge is to recognize the differences between lip shapes in one category. The purpose of this study is to recognize differences between bilabial consonants such as /p/, /b/, and /m/ in a word that composed of consonant/vowel called CV by computer vision. For the first time, this study attempts to distinguish these consonants. Proper pronunciations of words are required to identify consonants. Therefore, a database has formed based on the videos of the speech therapists. Generally, this kind of process is including 1-lip detection, 2- lip feature extraction, and 3- classification systems for the diagnosis of consonants. In this paper, consonants recognition in a category based on lip shape using the CLM algorithm for lip detection is presented. Geometric algorithms for feature extraction and DTW and equalizer as a classification system are proposed. Although this study is open because we could identify differences among consonants in just one class, we could reach remarkable CV video results for the first time. We could aim for acceptable results with reasonable accuracy for bilabial consonants detection. The principle purpose of this study is to improve lip-reading systems in security issues and help hearing-impaired people in interaction with their surroundings. The results of this paper can have a positive effect on speech systems.