Robust Fusion Model for Handling EMG and Computer Vision Data in Prosthetic Hand Control

Real-time surface electromyography (EMG) signals are commonly used in grasp classification for prosthetic hands. EMG signals, however, lack in getting comprehensive information and are susceptible to external influences. Integrating vision data into the grasp classification method alongside EMG signals could overcome this drawback. Toward this, we propose a new fusion method to combine computer vision and EMG signals to reduce the reliance solely on EMG signals, leading to an enhanced grasp classification system that is more accurate and robust. In our approach, an object detection algorithm is utilized to recognize various object types. Based on the identified object type and characteristics of the EMG signal, the model classifies between different grasp types. Object types and EMG feature set are given as input into a machine learning model, which performs the final grasp classification. We conducted a comparative analysis of the performance between our fusion model and existing ones, in which our model demonstrates superior performance compared to other fusion methods when assessed using the artificial neural network machine learning model. Notably, it exhibits exceptional efficacy when the classification accuracy of the object detection algorithm is low.