Dynamic Fingerspelling Recognition using Geometric and Motion Features

This paper presents the Australian sign language (Auslan) fingerspelling recognizer (APR): a system capable of recognizing signs consisting of Auslan manual alphabet letters from video sequences. The APR system uses a combination of geometric features and motion features based on optical flow which are extracted from video sequences. The sequence of features are then classified using hidden Markov models (HMMs). Tests using a vocabulary of twenty signed words showed the system could achieve 97% accuracy at the letter level and 88% at the word level by using a finite state grammar network and embedded training.