Semantic-based road environment recognition in mixed traffic for intelligent vehicles and advanced driver assistance systems

Comprehensive situational awareness is paramount to the effectiveness of higher-level functions of the intelligent vehicles and advanced driver assistance systems (ADASs). This paper addresses a hierarchical vision system designed for recognizing a number of objects of interest in mixed traffic, in which, the host vehicle have to drive inside the road boundary and interact with other road users. In the proposed system, the semantic knowledge of the scene is utilized to construct a graph. S tereo vision associated with the semantic graph is employed to seek the drivable road boundary in a Hidden Markov Model (HMM). The results are then used as the road contextual information for the following procedure, in which, particular objects of interest, including vehicles, pedestrians, motorcycles and bicycles, are recognized by using a multi-class object detector. Experimental results in various typical but challenging scenarios show the effectiveness of the proposed system.