An efficient vision-based traffic light detection and state recognition for autonomous vehicles

Traffic Light Detection(TLD) and understanding their state semantics at intersections plays a pivotal role in driver assistance systems and, by extension, autonomous vehicles. Despite of several reliable traffic light state detection approaches in literature, traffic light state recognition still remains an open problem due to outdoor perception challenge which includes occlusions, illumination and scale variations. This paper presents a vision-based traffic light structure detection and convolutional neural network (CNN) based state recognition method, which is robust under different illumination and weather conditions. In the first step, traffic light candidate regions are generated by performing HSV based color segmentation, which are then filtered out using shape and area analysis. Further, in order to incorporate the structural information of traffic light in diverse background scenarios, Maximally Stable Extremal Region (MSER) approach is employed, which helps to localize the correct traffic light structure in the image. To further validate the traffic light candidate regions, Histogram of Oriented Gradients (HOG) features are extracted for each region and traffic light structures are validated using Support Vector Machine (SVM). The state of the traffic lights are then recognized using CNN. To evaluate the performance of the proposed method, we present several results under a variety of lighting conditions in a real-world environment. Experimental result shows that the proposed method outperforms other vision based conventional methods under varying light and weather conditions.