Feature selection for visual gesture recognition using hidden Markov models

Hidden Markov models have become the preferred technique for visual recognition of human gestures. However, the recognition rate depends on the set of visual features used, and also on the number of states of the hidden variable. It is difficult to determine a priori the optimal set of features and number of states. We analyse experimentally the use of different features for gesture recognition in an office environment. We considered a set of seven gestures that include interaction with other objects, such as writing, using the mouse, opening a drawer, etc. We use a single camera to detect and track the hand of the user based on adaptive colour histograms. From tracking the hand in a video sequence we obtain several features. The features considered include position and velocity in polar and Cartesian coordinates, and the trajectory represented as a chain code. Given that these features are continuous, we discretized them into a set of symbols using vector quantization. We then tested the recognition rate using HMMs with different: (i) number of discrete symbols; (ii) number of hidden states, (Hi) combination of features. The results show a high variation on the recognition rate depending on these parameters, from below 50% to more than 95%. The best performance (97%) was obtained by using the magnitude and orientation in polar coordinates, 64 discrete symbols and 10 states.