Object-Based Visual 3D Tracking of Articulated Objects via Kinematic Sets

A theoretical framework based on robotics techniques is introduced for visual tracking of parametric non-rigid multi-body objects. It is based on an a-priori model of the object including a general mechanical link description. The objective equation is defined in the object-based coordinate system and non-linear minimization relates to the movement of the object and not the camera. This results in simultaneously estimating all degrees of freedom between the object's last known position relative to its previous position as well as internal articulated parameters. A new kinematic-set formulation takes into account that articulated degrees of freedom are directly observable from the camera and therefore their estimation does not need to pass via a kinematic-chain back to the root. By doing this the tracking techniques are efficient and precise leading to real-time performance and accurate measurements. The system is locally based upon an accurate modeling of a distance criteria. A general method is derived for defining any type of mechanical link and experimental results show prismatic, rotational and helicoidal type links. A statistical M-estimation technique is applied to improve robustness. A monocular camera system was used as a real-time sensor to verify the theory.