On Learning From Inaccurate and Incomplete Traffic Flow Data

Today, we live in an era where pervasive sensor networks both collect and broadcast rich digital footprints about the human mobility. However, most of this data often comes in an incomplete and/or inaccurate fashion. In this paper, we propose a knowledge discovery framework to handle such issues in the context of automatic incident detection systems fed with traffic flow data. This framework operates in three steps: 1) it clusters sensors with a novel multi-criteria distance metric tailored for this purpose, followed by a heuristic rule that labels the abnormal groups; 2) then, a spatial cross-correlation framework identifies seasonal and individual abnormal readings to perform a more fine-grained filtering; and 3) finally, we propose a novel fundamental diagram that discovers the critical density of a given road section/spot on a data-driven fashion that is resistant to both outliers and noise within the input data. Large-scale experiments were conducted over traffic flow data provided by a major Asian highway operator. The obtained results illustrate well the contributions of this framework: it drastically reduces the noise within the raw data, and it also allows determining reliable definitions of traffic states (congestion/no congestion) on a completely automated way.