Fast, integrated person tracking and activity recognition with plan-view templates from a single stereo camera

Plan-view projection of real-time depth imagery can improve the statistics of its intrinsic 3D data, and allows for cleaner separation of occluding and closely-interacting people. We build a probabilistic, real-time multi-person tracking system upon a plan-view image substrate that well preserves both shape and size information of foreground objects. The tracking's robustness derives in part from its "plan-view template" person models, which capture detailed properties of people's body configurations. We demonstrate that these same person models, obtained with a single compact stereo camera unit, may also be used for fast recognition of body pose and activity. Principal components analysis is used to extract plan-view "eigenposes", onto which person models, extracted during tracking, are projected to produce a compact representation of human body configuration. We then formulate pose recognition as a classification problem, and use support vector machines (SVMs) to quickly distinguish between, for example, different directions people are facing, and different body poses such as standing, sitting, bending over, crouching, and reaching. The SVM outputs are transformed to probabilities and integrated across time in a probabilistic framework for real-time activity recognition.