Quantifying Demonstration Quality for Robot Learning and Generalization

Learning from Demonstration (LfD) seeks to democratize robotics by enabling non-expert end-users to teach robots. However, most LfD techniques assume users provide optimal demonstrations, which may not be accurate. Demonstration quality plays a crucial role in robot learning and generalization. Hence, it is important to quantify the quality of the provided demonstrations before using them for robot learning. In this letter, we propose quantifying the generalizability of the demonstrations based on how well they perform in the learned task. The proposed approach is validated in a user study (N = 27). Participants with different robotics expertise levels were recruited to teach a PR2 robot a generic task (pressing a button) under different task constraints. They taught the robot in two sessions on two different days to capture their teaching behaviour across sessions. The task performance was utilized to classify the provided demonstrations into high-quality and low-quality sets. The results show a significant correlation between task performance and generalization performance across all participants. We also found that users clustered into two groups: Users who provided high-quality demonstrations from the first session (the
fast-adapters
), and users who provided low-quality demonstrations in the first session and then improved with practice (the
slow-adapters
). This approach for assessing demonstrations allows us to determine whether users require more training in order to provide high-quality demonstrations.