Finger-hand Rehabilitation using DNN-based Gesture Recognition of Low-cost Webcam Images

In this paper, an image-based gesture recognition system has been presented for finger hand rehabilitation using a low-cost camera. Since the goal is to set up a low-cost home rehabilitation system, the analysis of each finger should be easily possible for the user, and the user should be able to follow the information of the improvement process of one’s treatment. Hence, first, the models governing the movement angles of the fingers were established, and then some criteria have been developed to evaluate the improvement of the performance of the fingers. Finally, several deep-learning models were initially implemented to extract the hand gesture and model parameters and based on the experimental results, the MediaPipe framework was found suitable due to its precision and robustness to determine the finger angles in low quality images during each exercise.