Recognition of Traffic Lights in Live Video Streams on Mobile Devices

A mobile computer vision system is presented that helps visually impaired pedestrians cross roads. The system detects pedestrian lights in the environment and gives feedback about the current phase of the crucial light. For this purpose the live video stream of a mobile phone is analyzed in four steps: localization, classification, video analysis, and time-based verification. In particular, the temporal analysis allows us to alleviate the inherent problems such as occlusions (by vehicles), falsified colors, and others, and to further increase the decision certainty over a period of time. Due to the limited resources of mobile devices very efficient and precise algorithms have to be developed to ensure the reliability and the interactivity of the system. A prototype system was implemented on a Nokia N95 mobile phone and tested in real environment. It was trained to detect German traffic lights. For the prototype training and testing, we generated image and video databases including manually specified ground truth meta-data. These databases described in this paper are publicly available for the research community. Quantitative performance analysis is provided to demonstrate the reliability and interactivity of the prototype system.