A spatio-temporal feature descriptor for action recognition using feature relations

We propose a new feature descriptor for action recognition and crime detection using computation of feature relations in the spatio-temporal domain. The novelty of this approach is the computation of similarity of body-parts during movement of the body. Body-parts are represented by histograms of gradients which are dynamically positioned over the body using foreground segmentation. In order to capture moving body-parts over consecutive frames, an area around the current body-part is taken into consideration and the deformation of the body-part is computed over time using histogram similarity functions such as the chi-square distance. Primitives of deformations of body-parts are computed and assigned to specific actions, where the selection of primitives is done in a separate offline training stage. During recognition, a time window is used to sum up movement-primitives in order to compute confidences for different actions which have been trained beforehand. The computation of the overall system including our spatio-temporal relational feature model (STREL) is done with real-time performance, and our experiments show the potential of this descriptor for fast and robust action recognition.