Inverse Skill Learning From Demonstrations via Reinforcement Learning

In robotics, most semantically meaningful skills can be utilized to learn similar skills. Forward - inverse skill pairs are such a family of semantically meaningful skills where one skill can be used to learn the other. In this paper, we propose an architecture that is able to learn an inverse skill given the demonstrations of the forward skill using Reinforcement Learning (RL) coupled with Learning from Demonstrations (LfD). Our architecture uses learns and appropriate reward function for the inverse skill using the demonstrations of the forward skill, warm-starts the policy by learning to imitate the rewinded version of the demonstrations using Behaviour Cloning, and continues training with RL using the learned reward function. This approach mitigates the poor sample efficiency, tedious reward shaping and difficult high-dimensional exploration problems of RL while being able to create novel skills different from the given demonstrations, something that can’t be done by pure LfD.