Developing a Lightweight Rock-Paper-Scissors Framework for Human-Robot Collaborative Gaming

We present a novel implementation of a Rock-Paper-Scissors (RPS) game interaction with a social robot. The framework is tailored to be computationally lightweight, as well as entertaining and visually appealing through collaboration with designers and animators. The fundamental gesture recognition pipeline employs a Leap motion device and two separate machine learning architectures to evaluate kinematic hand data on-the-fly. The first architecture is used to recognize and segment human motion activity in order to initialize the RPS play, and the second architecture is used to classify hand gestures into rock, paper or scissors. The employed tabletop robot interacts in the RPS play through unique animated gestural movements and vocalizations designed by animators which communicate the robot's choices as well as cognitive reflection on winning, losing and draw states. Performance of both learning architectures is carefully evaluated with respect to accuracy, reliability and run time performance under different feature and classifier types. Moreover, we assess our system during an interactive RPS play between robot and human. Experimental results show that the proposed system is robust to user variations and play style in real environment conditions. As such, it offers a powerful application for the subsequent exploration of social human-machine interaction.