A Vision-Based Real Time Motion Synthesis System

This paper introduces the design of a real time vision-based motion synthesis system. the system requires user to wear the markers in a certain color. Based on that, several novel algorithms were used for feature detection and feature tracking under occlusion by estimating the velocity of missing features based on the prior, smoothness and fitness term. These algorithms ensured the accuracy and low computation cost of reconstruction of the 3D points in real time. the low-dimensional control signals from user's marker points were first used to construct a series of local models. When constructing these local models, we preprocess motion capture data to K-nearest neighborhood graph and store these data in KD-tree to ensure model building is real-time. in animation synthesis phase, we used an approach named locally weighted linear regression to synthesis the animation data closest to current pose. Results showed that our system can successfully synthesize three kinds of motion: running, walking and jumping.