A vision-based teleoperation method for a robotic arm with 4 degrees of freedom

The ability to remotely control a robotic arm through a human one is essential where human involvement is needed but physical presence is not possible. Control provided through vision-based approaches comes with advantage over non-vision schemes, as vision-based approaches are less intrusive. On the other hand, the problem of estimating the hand pose comes with numerous difficulties due to the nature of the hand itself. These difficulties include the high complexity of the hand and the presence of self-occlusions. In this paper, we provide a method for controlling a 4 degrees of freedom robotic arm. The arm is composed of 3 segments connected and controlled via servo motors. The end effector of the arm (the 3rd segment) is a gripper that simulates the human hand opening and closing movements. Features necessary to control the arm are 2D coordinates of the center of the human hand, its orientation, and its open/closed state. The results are reported and analyzed, limitations of the scheme are discussed, and possible future work is proposed.