Personalized Markerless Upper-Body Tracking with a Depth Camera and Wrist-Worn Inertial Measurement Units

A markerless motion capture technique is proposed based on a fusion between a depth camera (Kinect V2) and a pair of wrist-worn inertial measurement units (IMU). The method creates a personalized articulated human mesh model from one depth image frame and uses that model to improve the accuracy of the upper-body joint tracking. The IMUs are useful as an additional clue for the arm tracking, especially during an occlusion. An evaluation of the method against a marker-based system as a gold standard using data from 6 subjects is done. The result shows over 20% reduction in upper-limb joint position errors when compared to Kinect's skeleton tracking. All the collected data are calibrated, synchronized, and made publicly available for research purposes.