Dynamic road scene classification: Combining motion with a visual vocabulary model

The majority of studies in scene classification have focused on still images, ignoring potentially informative temporal cues. This paper explores the combination of multi-scale appearance and motion features for classification of scenes captured from a moving vehicle under real-world driving. The objective is to classify unknown scenes in one out of a set of predefined typical road scene classes that are learnt during training. The method is studying the performance of a state-of-the-art scene classification visual vocabulary model (known also as bag of features model) when appearance image features and video motion features are combined for SVM learning and classification. The sequence of scenes is captured from a moving vehicle equipped with a frontal camera sensor. Video driving data used for evaluation were available by two test vehicles (a passenger car and a truck) participating in the European interactIVe IP. It is shown that a notable performance increase is realized by appearance-temporal approach in comparison to purely appearance or purely temporal methods. The quantitative evaluation has been performed using manually annotated video sequences.