A Novel Multi-Sensor Fusion System with a Machine Learning-Based Human-Machine Interface for Automating Industrial Robots

This paper presents a novel method to control an industrial robotic arm using multiple sensors. This system consists of a hybrid brain activity and vision sensors that convey a human being's intention and visual perception. We fuse and analyze the data from those sensors using a machine learning-based approach to automatically guide the manipulator to a designated location. We believe that this Brain-Machine-Interface (BMI) can greatly alleviate the burdensome traditional method used to program a robot (greatly aids the end-user). We explore various machine learning and pattern recognition techniques as well as existing feature selection methods. Our experimental results show that the subject can control the robot to a destination of interest using a machine-robot-interface. We attain accuracy in the order of 99.4% when it comes to the desired motion and 100% for the case of deducing the desired characteristic (color) of the targeted object. In this work, we implemented and demoed our approach using a simple pick-and-place test case, a foundation for an underlying system that can be used to benefit people with restricted abilities and allow them to perform complex and robotics-related duties in an industrial setting.