Deep Learning based sEMG decoding for TelePresence device control

TelePresence devices enable their users to appear and move at distant locations. In this work, to achieve TelePresence, we installed a 360° camera on a mobile robot and streamed the video feed to a virtual reality headset. The user’s view field is outsourced, which is disadvantageous for traditional control methods. To overcome this issue, here we designed an sEMG-based control paradigm, enabling users to control a robot with hand gestures. We created a dataset of the necessary control commands, by recording the visual representation of the hand and the corresponding sEMG activity. The final dataset consisted of 5200 train and 400 validation samples. We investigated several neural network architectures to decode the biosignals and finally applied a CNN + LSTM Recurrent Neural Network to control a mobile robot by hand sEMG signals.