Combining ElasticFusion with PSPNet for RGB-D Based Indoor Semantic Mapping

Semantic segmentation has been a research focus of scene parsing and robotic manipulation. And increasing efforts are being made to empower geometry-solely robotic mapping with semantic sensing ability in complex spatial cognition tasks, the so-called semantic mapping. However, semantic mapping has not yet fully exploited the state-of-the-art of semantic segmentation and suffered limitations of application scenario, number of object category and accuracy of object detection. Towards a robust semantic mapping solution, we propose a RGB-D based fusion method that combines an improved ElasticFusion with PSPNet. PSPNet has the potential for breaking restrictions of limited object category in semantic segmentation of RGB images. As for map rendering, the improved ElasticFusion algorithm is validated for its robust performance in indoor scenes in terms of overall model accuracy and integral object shape geometry. Based on the two aforementioned modules, we fuses the semantic images and the depth images to generate 3D point cloud with semantic information. Experiments on the ICL-NUIM datasets have proven the feasibility of the proposed method.