Efficient policy search with a parameterized skill memory

Motion primitives are an established paradigm to generate complex motions from simpler building blocks. A much less addressed issue is at which level to encode and organize a library of motion primitives, and how to retrieve motion primitives from a library that fit a particular task. This paper proposes a parameterized skill memory, which organizes a set of motion primitives in a low-dimensional, topology-preserving embedding space. The skill memory acts as a pivotal mechanism that links low-dimensional skill parametrizations to motion primitive parameters and complete motion trajectories. The skill memory is implemented by means of a dynamical system which features continuous generalization of motion shapes. It is shown that the low-dimensional skill parametrization is beneficial for efficient, reward-based retrieval of motion primitives and simplifies the shaping of reward functions. The excellent generalization of motion shapes by parameterized skill memories from few training examples is demonstrated in a bimanual manipulation task with the humanoid robot iCub.