Gesture based robot programming using ROS platform

In the digital twin concept, the true copy of the real factory floor must be mirrored in the virtual world not only by its geometry, but also by the behavior of the devices. A pre-programmed production line is common to represent and simulate in the virtual world, but to bring in the human-machine interaction is more difficult. This kind of interaction is expected and necessary in the case of collaborative robots, so it must be integrated in the virtual world too. This paper presents a solution for the human-robot interaction (ex: robot program teaching) based on the LeapMotionhand-tracking sensor implemented on Robot Operating System (ROS). By realizing the connection between the ROS and the robot simulation software ABB RobotStudio, it is possible to transformhuman gesture information into robot commands. Using the same principle, the real robot can be teached as well.