Highway scene analysis in hard real-time

A hard real-time vision system has been developed that analyses color videos taken from a car driving on a highway. The system uses a combination of color, edge, and motion information to recognize and track the road boundaries, lane markings and other vehicles on the road. Cars are recognized by matching templates that are cropped from the input data online, by detecting image features, and by evaluating how these features relate to each other. Cars are also recognized by temporal differencing and by tracking motion parameters that are typical for cars. The system recognizes and tracks road boundaries and lane markings using a recursive least squares filter. Experimental results demonstrate robust, real-time car recognition and tracking over thousands of image frames.