See What I See: Enabling User-Centric Robotic Assistance Using First-Person Demonstrations

We explore first-person demonstration as an intuitive way of producing task demonstrations to facilitate user-centric robotic assistance. First-person demonstration directly captures the human experience of task performance via head-mounted cameras and naturally includes productive viewpoints for task actions. We implemented a perception system that parses natural first-person demonstrations into task models consisting of sequential task procedures, spatial configurations, and unique task viewpoints. We also developed a robotic system capable of interacting autonomously with users as it follows previously acquired task demonstrations. To evaluate the effectiveness of our robotic assistance, we conducted a user study contextualized in an assembly scenario; we sought to determine how assistance based on a first-person demonstration (user-centric assistance) versus that informed only by the cover image of the official assembly instruction (standard assistance) may shape users’ behaviors and overall experience when working alongside a collaborative robot. Our results show that participants felt that their robot partner was more collaborative and considerate when it provided user-centric assistance than when it offered only standard assistance. Additionally, participants were more likely to exhibit unproductive behaviors, such as using their non-dominant hand, when performing the assembly task without user-centric assistance. CCS Concepts • Human-centered computing → Collaborative interaction; • Computer systems organization → Robotics; • Computing methodologies → Vision for robotics. ACM Reference Format: Yeping Wang, Gopika Ajaykumar, and Chien-Ming Huang. 2020. See What I See: Enabling User-Centric Robotic Assistance Using First-Person Demonstrations. In Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction (HRI ’20), March 23–26, 2020, Cambridge, United Kingdom.. ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/ 3319502.3374820