Camera-Lidar-Based Terrain Multi-type Classification Using Both Spatial and Histogram Features of Lidars

Terrain classification is a necessary and difficult task for all off-road robots. Most existing methods use images or proprioceptive sensors for recognition. However, while taking proprioceptive sensors as input requires more actual contact (which also means risks), images are susceptible to ambient light change. We also found that a combination of multiple optical channels in Lidars is quite useful for the identification of terrain types. In this paper, we designed a Lidar-camera fusion system fully utilizing information from Lidars and fusing depth to images to classify different types of terrain. The proposed network was tested on the dataset which includes asphalt, brick, cement, cobblestone, grass, and gravel. The accuracy achieves 99.471%, which shows its efficiency.