Visual activity recognition based on depth contour image

In this paper, a novel visual activity recognition algorithm based on the depth contour image and nearest neighbor (NN) classifier is presented, where the depth contour image is acquired by solving algebraic matrix equations in terms of the newly-defined Spatial Motion Accumulative Image (SMAI) and Temporal Motion Accumulative Image (TMAI). More precisely, firstly, SMAI and TMAI are defined based on the binary image sequences of action video segments; secondly, the pixel values in SMAI are re-evaluated by the average traveling time for (virtual) free particles from the corresponding pixel positions to the contour border with the so-called TMAI-determined speed, which leads to the proposed depth contour image; thirdly, the principal component analysis (PCA) is applied to extract the feature vectors from the depth contour image to characterize human actions; finally, the NN classifier is used to recognize the human actions. Experimental results on the open Weizmann activity database confirm the expected recognition performance of the proposed algorithm.