Vision system for identifying road signs using triangulation and bundle adjustment

This paper presents improvements made to an automated machine vision system that identifies and inventories road signs. The system processes imagery from the Kansas Department of Transportation's road profiler that captures images every 26.4 feet on highways through out the state. The initial system processed images using a computationally efficient K-means clustering algorithm twice, first on the original image and then again on a difference image to segment the images into objects. Next, object segments were classified based on their size and color. An additional classification step was applied examining the frame to frame trajectory that objects take through the field of view. This technique represented a crude form of triangulation. It was quite effective for signs along straight highways, but often failed along curves when trajectories deviate from the norm. This paper describes how full triangulation was implemented with bundle adjustment to determine the object's physical location relative to the road profiler. Object locations are then added to the list of criteria determining classification. As with the original size and color classifiers, a representative image set was segmented and manually labeled to determine a joint probabilistic model characterizing the expected location of signs. Receiver operating characteristic curves were analyzed to adjust the thresholds for class identification. The improved sign inventory system was tested and its performance characteristics are presented.