A Cross-Space Hand Motion Transfer Method for Dexterous Hand Teleoperation

Teleoperation, as one of the core technologies in the field of robot imitation learning, currently includes methods based on visual perception, handheld gripping devices, data acquisition gloves, and more. However, when it comes to dexterous hand teleoperation, these methods often encounter challenges such as motion distortion, biomechanical skeletal structure confusion, and accumulated mapping errors of feature points. To address these challenges, this paper proposes a cross-space hand motion transfer method for dexterous hand teleoperation. It utilizes a binocular vision system in the physical space to perceive the movement trajectories of hand feature points, employs a robust state estimation method based on the Kalman filter for noise suppression, and subsequently transfers the processed feature point trajectory sequence to the dexterous hand model in the Unity virtual space. Under biomechanical constraints, a feature point mapping matrix is constructed to replicate hand motions in the virtual space. Finally, inverse kinematics under biomechanical constraints is used to solve the joint angle sequence, which is then applied to the physical dexterous hand, achieving high-precision teleoperation. Experiments show that the proposed hand motion transfer method achieves a bone length error of less than 2.9 mm in static pose reproduction and an average success rate of 89.5 % in dynamic tasks, significantly outperforming the baseline system driven purely by visual feedback, which achieved a success rate of 75.1 %. This provides an efficient and stable technical method for dexterous hand teleoperation.