Target prediction and temporal localization of grasping action for vision-assisted prosthetic hand

With the development of shared control technology for humanoid prosthetic hands, more and more research is focused on vision-based machine decision making. In this paper, we propose a miniaturized eye-in-hand target object prediction and action decision-making framework for the humanoid hand “approach-grasp” sequence. Our prediction system can simultaneously predict the target object and detect temporal localization of the grasp action. The system is divided into three main modules: feature logging, target filtering and grasp triggering. In this paper, the optimal configuration of the hyper-parameters designed in each module is performed experimentally. We also propose a prediction quality assessment method for “approach-grasp” behavior, including instance level, sequence level and action decision level. With the optimal hyper-parameter configuration, the predicting system perform averagely to 0.854 at instance prediction accuracy (IP), 0.643 at grasp action prediction accuracy (GP). It also has good predictive stability for most classes of objects with number of predicting changes (NPC) below 6.