Facial feature tracking and expression recognition for sign language

Expressions carry vital information in sign language. In this study, we have implemented a multi-resolution Active Shape Model (MR-ASM) tracker, which tracks 116 facial landmarks on videos. Since the expressions involve significant amount of head rotation, we employ multiple ASM models to deal with different poses. The tracked landmark points are used to extract motion features which are used by a Support Vector Machine (SVM) based classifier. We obtained above 90% classification accuracy in a data set containing 7 expressions.