Self-Supervised Traversability Prediction by Learning to Reconstruct Safe Terrain

Navigating off-road with a fast autonomous vehicle depends on a robust perception system that differentiates traversable from non-traversable terrain. Typically, this depends on a semantic understanding which is based on supervised learning from images annotated by a human expert. This requires a significant investment in human time, assumes correct expert classification, and small details can lead to misclassification. To address these challenges, we propose a method for predicting high- and low-risk terrains from only past vehicle experience in a self-supervised fashion. First, we develop a tool that projects the vehicle trajectory into the front camera image. Second, occlusions in the 3D representation of the terrain are filtered out. Third, an autoencoder trained on masked vehicle trajectory regions identifies low- and high-risk terrains based on the reconstruction error. We evaluated our approach with two models and different bottleneck sizes with two different training and testing sites with a four-wheeled off-road vehicle. Comparison with two independent test sets of semantic labels from similar terrain as training sites demonstrates the ability to separate the ground as low-risk and the vegetation as high-risk with 81.1% and 85.1% accuracy.

I. Introduction

Fast autonomous off-road and off-trail driving requires robust and accurate perception and understanding of the unstructured terrain in which the vehicle is navigating. It also often necessitates traversing through different surface types which could include different types of traversable and non-traversable vegetation or surfaces with different properties such as sand or soil. Therefore, it is crucial to understand what surfaces pose a low risk to the vehicle and which areas have a higher risk. However, many geometric-based approaches [1], [2] typically require additional terrain classification algorithms based on supervised learning to capture the variety of terrain [3]–[5]. Semantic labeling requires a significant investment of human time to manually annotate the data. Additionally, the boundary between many different classes, especially vegetation types, can be difficult and laborious to determine for human annotators, since some of the largest public data sets contain data from only a single natural environment [5]–[7]. Therefore, it would be ideal to learn which terrain in any environment is traversable and which is non-traversable using only previous experiences of the vehicle via a self-supervised approach [8]–[10]. Further-more, to drive fast and with a highly capable vehicle such as the Polaris RZR (Figure 1) in our case, different semantic classes could pose different risks compared to other vehicles. In order to scale a perception system to handle a wide variety of natural terrains and vehicle risk tolerances, efficient self-supervised learning techniques are needed.