Road Roughness Estimation for Intelligent Vehicles Based on SNE and Semantic Segmentation

The rapid development of unmanned systems benefits from the improvement of perceptual capabilities. However, there is a lack of road roughness estimation for intelligent vehicles. This paper proposes a real-time road roughness estimation method, which can be applied by intelligent vehicles to obtain the specific position and height of the road patches. The binocular camera mounted on the intelligent vehicle can capture RGB images and parallax information. The images are input into the devised lightweight semantic segmentation model, which can accurately recognize each category on the road. Based on the Surface Normal Estimator (SNE), the height of each point in the Region of Interest (ROI) above the road is estimated. After multi-frame fusion, filtering processing and other operations, the specific height and position of the bumps in the road are obtained. Based on the combination of VGG and FPN, the proposed method converts an image to several images with different sizes as inputs and utilizes the attention mechanism and the adaptive loss function of multiple branches. We further validate our approach on a real-world dataset. The experimental results indicate that the mean Intersection over Union (mIoU) of the devised semantic segmentation model can achieve 0.686, and the frames per second (FPS) of the integrated method can reach 78. In addition, the proposed method can estimate the height of the road bumps with 91% accuracy, which can be leveraged to estimate the road roughness for intelligent vehicles accurately in real-time.