3-D human body tracking from depth images using analysis by synthesis

A general method to estimate MPEG-4 body animation parameters (BAPs) from depth images by using an analysis-by-synthesis technique is presented. A generic body model is first adapted to the specific person geometry. Then, the mean square error between the synthetic depth image, produced by model rendering, and the original depth image is minimized using the downhill simplex minimization method. Use of this error norm is seen to yield improved results, when compared to two alternative error norms, which were also evaluated. Results are presented for the specific application, where six animation parameters of the arm and the palm are estimated. In this application, an initial estimate is obtained by applying an expectation-maximization (EM) algorithm, which identifies three arm parts and two joint positions (elbow, wrist). A significant advantage of this procedure is that the resulting information can be exploited for reducing the search space and for automatic scale adaptation of each body part.