Gesture Based Symbiotic Robot Programming for Agile Production

Agile production lines call for an effective and intuitive way of programming robots. However, traditional approaches rely on providing low-level instructions using either a script-based language or a graphical user interface to export low-level instructions. This, however, can be tedious for assembly tasks. In this work, we present an approach that generates low-level robot control commands from highly abstract communicative hand gestures. In contrast to other works, we use several abstraction layers to generate such commands with as little user input as possible. For this, we use a body-attached multi-sensor setup consisting of a pressure band, a smart glove, EMG and IMU units. Their combined signals define a multi-dimensional vector per time step. We use a Recurrent Neural Network to infer the gesture class from the pre-processed data stream. From these user inputs we generate a set of symbolic spatial relations describing the assembly process. This formal description is then used to select and execute robot skills such as grasping. Hence, we reduce the ambiguity of abstract instructions in several steps and allow for effective gesture-based robot programming. In our work we give insights in defining and detecting such gestures. In addition, we illustrate the functionality of the whole system at real-world examples.