Pointing Gestures for Human-Robot Interaction with the Humanoid Robot Digit

We present and evaluate a pointing-gesture-based, bilateral human–robot interaction (HRI) with the humanoid robot Digit. Recently, humanoid robots have become more powerful and available, but they are still not helpful and accessible to non-technical users without programming experience. Therefore, we propose a pointing gesture-based interaction modality based on monocular video input. The pointing gestures are extracted using a pre-trained pose estimation model and, together with a verbal dialogue, serve as an exemplar interaction behavior for the task of moving heavy objects using the Digit. The interaction is evaluated quantitatively and qualitatively in a user study. The results of the study show promising results in terms of usability and implementation challenges for further research in this area.