Simultaneous learning of the structure and kinematic model of an articulated body from point clouds

We present an algorithm that simultaneously identifies the structure and learns the kinematic model of an articulated body from point cloud data and joint angle data with minimal prior assumptions. The robot arm is represented by a point cloud. The structure is identified by segmenting the point cloud to subsets corresponding to different links according to a minimum matching error criterion that includes spatial continuity. The kinematic model is represented as a set of rigid transformations, which are learned by a stochastic gradient decent-based iterative closest point (ICP) algorithm. The learned transformations are then used to infer the structural relationship between segments. We validate the experiment on both synthetic data and data acquired by a Kinect depth sensor viewing a robot arm moving three of its joints. Our results demonstrate that point cloud can be correctly segmented into different rigid parts; the rigid transformations associated with each part can be learned and the kinematic structure can be correctly inferred. Our proposed approach makes a step toward a unified framework for robot self-discovery and learning forward kinematic models purely with data from depth sensors and joint angle encoders.