A Fusion Model for Road Detection based on Deep Learning and Fully Connected CRF

This paper presents a road detection model based on deep learning and fully connected condition random field to fuse image and point cloud data. Firstly, a convolutional neural network is trained to extract multi-scale features of the image. And a point-based deep neural network is trained to extract the multi-scale features of the point cloud. Secondly, the point cloud data is projected to the image plane. The probability maps of image and point cloud in the image plane are obtained by their corresponding multi-scale features, respectively. Thirdly, a Markov-based up-sampling method is used to get a dense height image from a sparse one which is from the point cloud data. A fully connected condition random field model based on the outputs of the two networks and the height image is constructed on the image plane. Finally, the fusion model is effectively solved by the mean-field approximate algorithm. Experiments on KITTI Road dataset show that the proposed model can effectively fuse the image and the point cloud data. Furthermore, the fusion model can also exclude the shadows, road curbs and other interferences in complex scenes.