Human action recognition by sequence of movelet codewords

An algorithm for the recognition of human actions in image sequences is presented. The algorithm consists of 3 stages: background subtraction, body pose classification, and action recognition. A pose is represented in space-time, called 'movelet'. A movelet is a collection of the shape, motion and occlusion of image patches corresponding to the main parts of the body. The (infinite) set of all possible movelets is quantized into codewords obtained by the vector quantization. For every pair of frames each codeword is assigned a probability. Recognition is performed by simultaneously estimating the most likely sequence of codewords and the action that took place in a sequence. This is done using hidden Markov models. Experiments on the recognition of 3 periodic human actions, each under 3 different viewpoints, and 8 non-periodic human actions, are presented. training and testing are performed on different subjects with encouraging results. The influence of the number of codewords on the algorithm performance is studied.