Towards a Multimodal System combining Augmented Reality and Electromyography for Robot Trajectory Programming and Execution

Programming and executing robot trajectories is a routine manufacturing procedure. However, current interfaces (i.e., teach pendants) are bulky, unintuitive, and interrupts task flow. Recently, augmented reality (AR) has been used to create alternative solutions. However, input modalities of such systems tend to be limited. By introducing the use of electromyography (EMG), we have created a novel multimodal wearable interface for online trajectory programming and execution. Through the use of EMG, our system aims to bridge the user’s force activation to the robot arm force profile. Our proposed system provides two interaction methods for trajectory execution and force control using 1) arm EMG and 2) arm orientation. We compared these methods with a standard joystick in a user study to test their usability. Results show that proposed methods have increased physical demands but yield equivalent task performance, demonstrating the potential of our proposed interface to provide a wearable alternative solution.