An End-to-End Pothole Detection and Avoidance System for Autonomous Ground Vehicles

Contemporary research in autonomous driving has primarily focused on improving perception and planning subsystems to allow vehicles to better perceive and react to environmental uncertainties, but mostly in countries with well-maintained road infrastructures. Given that potholes are common structural road defects, an autonomous vehicle must be able to perceive and plan around potholes to avoid vehicular damage. This paper proposes an end-to-end pothole detection and avoidance system that runs in real-time consisting of an instance segmentation model running on a stereo camera and a path planner designed to execute four different maneuvers: (1) pothole avoidance, (2) alignment between the centers of the vehicle and the pothole, (3) deceleration and slowly passing over the pothole, and (4) stopping before the pothole. The overall system is deployed on an Intel Core i7-97 50H CPU laptop with an NVidia GTX 1650 GPU. All four maneuvers are executed on the NV-X1 platform, and experimental results demonstrate the proposed system's ability to segment pothole instances in 3D, plan an appropriate response maneuver, and successfully execute each maneuver.