Hybrid Vision Based Reach-to-Grasp Task Planning Method for Trans-Humeral Prostheses

This paper proposes a hybrid vision-based reach-to-grasp task planning method for transhumeral prostheses exploiting both vision and electromyography (EMG) signals. The hybrid method mainly consists of 2-1/2D visual servoing module and EMG-based module. The visual servoing intends to align the object on to the center of the palm while correcting its orientation. EMG signals extracted from the remaining muscles of the disabled arm due to amputation are used to control the elbow flexion/extension (FE). While using the 2-1/2D visual servoing module, the object reaching algorithm changes the elbow FE angle to reach the palm toward the object of interest. Initially, the EMG-based module controls the elbow FE. Once an object is detected, the EMG signals emanating from the arm muscles generate a reach request. This process then activates the visual servoing module to bring the palm toward the object. Since both EMGbased module and the visual servoing module are producing elbow FE angles while reaching toward an object, these two modules are integrated to obtain a resultant angle for elbow FE. Experiments are conducted using a simulation environment and a prosthesis to validate the proposed task planning method. The EMGbased module is capable of following the natural elbow FE motion. Moreover, the task planning method is capable of driving the prosthesis toward the object with proper orientation.