Hierarchical Segmentation of Human Manipulation Movements

This paper introduces a segmentation algorithm which splits complex human manipulation movements into movement segments and automatically groups these into labeled actions. With this hierarchical algorithm the basic movement identities, which we call building blocks, as well as their concatenation to more complex actions can be identified in one handed as well as dual arm human manipulation movements. The algorithm can be used, e.g., in robotic applications such as imitation learning, in which human movement examples are directly used to generate robotic behavior. In this paper, we present two variants of the hierarchical segmentation algorithm, one supervised approach which requires a small number of pre-labeled movements as training data, as well as an approach which uses unsupervised algorithms to group building block segments which belong to the same movement. In both variants, the building block movements are detected based on the velocity of the hand(s), using the velocity-based multiple change-point inference algorithm. We evaluate both methods on human manipulation movements recorded from several participants with a marker-based motion tracking system. The first evaluations are done on simple one-handed point-to-point movements, followed by an evaluation on a complex dual arm manipulation task. The results show, that the presented approaches are able to identify basic movements as well as their concatenations into more complex, labeled actions.