Robot arm teleoperation via RGBD sensor palm tracking

In this paper we present a simple and intuitive approach to teleoperating a 3 or higher degrees-of-freedom (DOF) robotic arm in Cartesian space. Using an RGBD camera, we retrieve the position of the user's palm. This position is then translated into the desired robotic arm position, which is then used as an input to a control loop. The entire system is implemented in the Robotic Operating System, enabling simple functionality transfer to any compatible robotic arm. The system was tested on the Kinova Jaco 6DOF robotic arm with the aim of using it for object manipulation. We use the inverse kinematics for calculating the joint rotation velocities required for following the Cartesian path of the human hand. The resulting joint velocities are then sent to the robotic arm control interface which then passes commands to the pertaining API. Results corroborate the validity of the proposed approach for robotic arm teleoperation, opening the possibility for many potential applications.