Spatial scene representation and navigation in a mobile robot using RGB-D camera

The paper presents an algorithm for creating 3D scene representation to be used in a mobile robot. The operation of the algorithm is based on data collected with an RGB-D camera. The RGB scene view is further processed with instance segmentation algorithm based on convolutional neural networks to obtain binary masks of the recognized objects. The masks are matched to the depth view of the scene, which gives point clouds reflecting 3D shape of objects. Then the point clouds are averaged to reduce all objects into single points. In this way we obtain minimalistic representation of a scene, which is easy for storing and processing even in case of very large and complex scenes. This scene representation is aimed to support a mobile robot in recognizing scenes, and finding locations. The experiments were based on the Parallax Arlo robotic platform, and the Microsoft Kinect for Azure camera.