A novel human-robot interaction system based on 3D mapping and virtual reality

Aiming at improving the level of the Human-Robot Interaction (HRI), a novel HRI system based on 3D mapping and virtual reality (VR) is proposed, in the background of rescue robots. In the HRI system, the rescue robot constructs a 3D map in the complex post-disaster environment based on a multi-line LiDAR and an inertial measurement unit (IMU) in real time. The map is represented using 3D-NDT, which is incrementally transmitted to the VR system. In the VR side, the human operator uses interactive devices to generate commands to control the robot's movement remotely, in order to perform the rescue task. Using the proposed HRI system, the operator can obtain a strong sense of immersion, thus can better understand the robot's working environment. In addition, as a new way of human-robot interaction, the HRI system provides new ideas to improve the natural level of the interaction between the robot and the human operator.