A Solution of Human-Computer Remote Interaction with Tactile Feedback

In human-computer interaction, we need to use human's multiple sensory channels and motion channels to improve its naturalness and efficiency. These channels include voice, handwriting, posture, vision, expression, etc. Visual-based human-computer interaction systems are often mentioned in the literature. The system uses a camera to monitor and track the operator's posture, and sets corresponding commands to control the movement of the robot. On top of that, we add tactile feedback to allow the robot to accomplish more tasks. In the vision system, first we obtain color images and depth images through a depth camera. Secondly, we estimate the human posture through OpenPose. Finally, we map the motion of the human arm to the motion of the robotic arm. In the tactile system, first we control the movement of the magic hand through the exoskeleton. Secondly, we obtain the force values of the fingers through the magic hand. Finally, we provide force feedback to each finger through the exoskeleton.