PAVEMENT: Passing Vehicle Detection System with Autonomous Incremental Learning using Camera and Vibration Data

Systems that detect vehicles passing through roads play a significant role in ITS (Intelligence Transport Systems), due to their wide applicability to traffic monitoring and analysis for road construction/repair planning, congestion, and prediction. Among various systems using cameras, doppler sensors etc., a system that uses road vibration to detect passing vehicles is promising since it has advantages in terms of weather conditions and deployment/operation costs. However, it suffers from the human labor to prepare ground truth labels for training models. In this paper, we propose PAVEMENT, a novel Autonomous Incremental Learning based traffic-census sensor system using a piezoelectric vibration sensor and a video camera without human intervention. PAVEMENT consists of two models: the video-based model which detects vehicles by using bounding boxes (detected by YOLOv3 and DeepSORT) and the vibration-based model which uses road vibrations to detect passing vehicles. To reduce the burden of collecting ground truth labels, we apply linear discriminant analysis and incremental learning to train the vibration-based model by using the result of the video-based model as ground truth. Once the vibration-based model is trained, it can be used for traffic census on roads without the video camera for various conditions (weather, lighting, and other environmental factors). We collected the video and vibration data of more than 4,000 passing vehicles on roads in different places and applied our method to the data. As a result, PAVEMENT achieved over 98.4% accuracy and 98.0% f1-score in detecting passing vehicles using the model trained with 15 incremental learning steps in 1 minute interval.