The Design of an Augmented Reality System for Urban Search and Rescue

Most robots in urban search and rescue (USAR) fulfill tasks teleoperated by human operators (Such as controlling robot movement to avoid obstacles and explore unknown environments). The operator has to know the location of the robot and find the position of the target (victim). This paper presents an augmented reality system using a Kinect sensor on a customly designed rescue robot. Firstly, Simultaneous Localization and Mapping (SLAM) using RGB-D cameras is running to get the position and posture of the robot. Secondly, a deep learning method is adopted to obtain the location of the target. Finally, we place an AR marker of the target in the global coordinate and display it on the operator's screen to indicate the target even when the target is out of the camera's field of view. The experimental results show that the proposed system can be applied to help humans interact with robots.