Regional Semantic Learning and Mapping Based on Convolutional Neural Network and Conditional Random Field

Semantic information of objects and environment is a basis for robots to effectively complete complicated tasks. However, scene recognition methods based on image descriptors or convolutional neural networks often have misclassifications in actual environment. One reason is that highly complex environment has inconspicuous physical boundaries. In this paper, we propose a regional semantic learning method based on convolutional neural networks (CNNs) and conditional random fields (CRFs). The method combines global information obtained by scene classification network and local object information obtained by object detection network to train a CRF scene recognition model. Then the model can be used to infer the semantics of the region. After that, the regional semantic information is applied to build a sparse semantic map based on ORB-SLAM2. The proposed method was tested on a self-built environment dataset which contains four regional categories. Experimental results have demonstrated that the proposed method is effective and can obtain better classification results.