Real-Time Human Pose Estimation as a Cost-Effective Solution for the Teleoporation of a 6-Axis Cobot Arm

This paper explores the application of BlazePose, a monocular human pose estimation (HPE) model, within a teleoperation framework for a UR5 six-axis robot. Achieving teleoperation with only a single RGB camera and a device without a powerful GPU will improve accessibility and cost effectiveness of teleoperation solutions. This study evaluates the 2D pose estimation capabilities of BlazePose for robotic teleoperation tasks. Given the necessity of manipulating the UR5 in three- dimensional space, we implement a 2D-based controller that translates the teleoperator's 2D right hand position within a configurable hand workspace to the corresponding position of the robot's Tool Centre Point (TCP) within the robot's available workspace along two dimensions. The left hand is then utilised for controlling the robot's motion along the third dimension and operating the attached OnRobot RG2 gripper during the pick- and-place task. Additionally, we explore an alternative control paradigm utilising the 3D pose estimation of BlazePose for a more intuitive controller. Two experiments are conducted: the pick-and-place task to assess the 2D-based controller in common robotic tasks, and a hold position task. The hold position task aims to assess the amount of excess movement attributable to the HPE model when utilising the 3D-based controller. The results reveal that while the 2D pose estimation capabilities enable effective teleoperation, the utilisation of 3D estimation results in poor translation to robot control and significant excess motion. These findings underscore the importance of accurate depth estimation in 3D HPE models for precise and reliable teleoperation.