ActIVO: An Active Perception Framework for Skill Transfer Through Iterative Visual Observations

Learning by demonstration (LbD) through visual observation is widely used in literature for tackling the problem of robot programming. However, identification and exact localization of the point of interest (e.g. human hand) during the demonstration, in most of the cases, comes with a non-negligible uncertainty, introduced by the detection algorithm and the sensor characteristics. In this work, we propose an active perception framework for gaining the maximum information during iterative demonstrations performed by a human, towards LbD. The method considers an in-hand camera and it is based on a covariance weighted pseudo-inverse estimator that accounts for the non-homogeneous uncertainty of the sensor. The proposed framework, coined as “ActIVO”, is experimentally validated in two scenarios, utilizing a UR5e robotic manipulator and an in-hand RealSense RGB-D camera.