An Offline-Merge-Online Robot Teaching Method Based on Natural Human-Robot Interaction and Visual-Aid Algorithm

This article proposes an offline-merge-online robot teaching method (OMORTM). Specifically, a virtual-real fusion interactive interface (VRFII) is first developed by projecting a virtual robot into the real scene with an augmented-reality (AR) device, aiming to implement offline teaching. Second, a visual-aid algorithm (VAA) is proposed to improve offline teaching accuracy. Third, a gesture and speech teaching fusion algorithm (GSTA) with the fingertip tactile force feedback is developed to obtain the natural teaching pattern and improve the interactive accuracy of teaching the real or virtual robot. More specifically, through the VRFII, the operator can use the GSTA and the VAA to teach the virtual robot naturally and safely, and then the real robot reproduces the motion of the virtual robot. Therefore, OMORTM enables the teaching results to be quickly verified while ensuring the operator's safety and avoiding damage to the robot or workpiece. A series of experiments were conducted to validate the practicality and effectiveness of OMORTM. The results show that by effectively combining the offline and online, OMORTMprovides accurate robotic teaching processes, suitable for nonprofessionals.