A hybrid stochastic-connectionist architecture for gesture recognition

An architecture for the recognition of dynamic gestures is described. The system implemented is designed to take a sequence of images and to assign it to one of a number of discrete classes where each of them corresponds to a gesture from a predefined vocabulary. The classification task is broken down into an initial preprocessing stage following by a mapping from the preprocessed input variables to an output variable representing the class label. The preprocessing stage consists of the extraction of one translation and scale invariant feature vector from each image of the sequence. Further we utilize a hybrid combination of a Kohonen self-organizing map (SOM) and discrete hidden Markov models (DHMM) for mapping an ordered sequence of feature vectors to one gesture category. We create one DHMM for each movement to be detected. In the learning phase the SOM is used to cluster the feature vector space. After the self-organizing process each codebook is quantized into a symbol. Every symbol sequence underlying a given movement is finally used to train the corresponding Markov model by means of the nondiscriminative Baum-Welch algorithm, aiming at maximizing the probability of the samples given the model at hand. In the recognition phase the SOM transforms any input image sequence into one symbol sequence which is subsequently fed into a system of DHMMs. The gesture associated with the model which best matches the observed symbol sequence is chosen as the recognized movement.