Real-Time Physics-Based Removal of Shadows and Shading From Road Surfaces

We present a real-time physics-based system for generating an illumination free representation of road surfaces that maintains the distinction between asphalt and painted road markings. Cast shadows on road surfaces can create false features and modify the color of road markings, potentially masking important information for vehicle vision systems. We demonstrate a novel method for identifying the relative spectral properties of the direct and ambient illumination conditions and for using that to create an illumination-free 2D chromaticity space in log RGB. We then show how that representation can be used to generate an illumination-free greyscale representation that distinguishes road, white paint, and yellow paint, making it suitable for further analysis and classification. The entire process runs faster than 30Hz on current automotive-grade embedded processing systems. We evaluate the system on a paint detection task, comparing two types of learned classifiers, random forests and convolutional neural networks. For each type, one classifier is trained on the original images, and the other is trained on the illumination-free greyscale output. The classifiers are of identical complexity and trained on the same size data set. For both types, the classifier trained on the illumination-free outputs performs better, even on images with no cast shadows. The gap in performance is indicative of the cost of forcing a classifier to learn a task in the presence of the confounding illumination signal.