Virtual Reality Capabilities for Robot Programming by Demonstration

This paper presents a novel Virtual Reality (VR)based framework for programming collaborative robots by demonstration, focusing on industrial painting applications. The framework enables painting specialists to program industrial robots intuitively, without elevated programming knowledge. Robot programs are created just by recording painting trajectories simulated in a virtual environment using VR headset and controls. This novel approach enhances humanrobot interaction, improves painting efficiency and reduces material waste. By employing virtual reality, human and robot interaction can be combined to achieve a more efficient ecosystem for creating painting programs for collaborative robots. The virtual reality deployed through mixed reality devices improves the interaction with digital information, increasing the level of perception required for robot programming. The collected data include painting pressure, the orientations and the velocity of the painting paths. The proposed framework has been developed for the Meta Quest 2 mixed reality headset, engine in Unity using TCP/IP sockets for UR10 collaborative robot, and applied in a furniture painting case study. The robot programs are generated in a simulation environment thanks to programming by demonstration and transformed to be executed in UR script format in a ROS environment. This novel approach uniquely combines VR and digital twins to transform craftmanship work into robotic programs. The present framework has effectively captured human skills and transferred them to the painting collaborative robot program.