Navigating Off-Roads: Using Deep Neural Networks for Rock Detection in Off-Road Autonomous Driving with Unimog

An off-road environment is characterized by multiple objects, often ambiguous in nature. The challenges associated with autonomous navigation in off-road environments are numerous, and detecting rocks, a critical element, can be particularly difficult due to their variability. This paper provides insights into the challenges associated with rock detection in Earth's off-road environments and the capabilities of Deep Neural Networks (DNNs) (with a considerably smaller dataset) to perform this task. Three state-of-the-art instance segmentation networks are trained on the rock dataset and compared. The best network is then implemented (in Unimog's robotic framework) and 3D bounding boxes (in laser point cloud) are generated. The generated boxes can be used to create a 3D rock dataset capturing their characteristics in more detail. This pipeline serves as a key component of the environment perception package, ensuring safe navigation of the Unimog U5023 in off-road environments. The networks are further tested against different domains and qualitative and quantitative results are provided through real-world driving tests.