Vietnamese vehicles speed detection with video-based and deep learning for real-time traffic flow analysis system

In this paper, we have developed a system to leverage traffic surveillance cameras to detect vehicle speed. In this system, we use a detection-based tracking paradigm for multiple object tracking then speed is estimated. First, YOLOv4 with transfer learning is applied for vehicle detection, a comparative analysis is carried out to choose trackers that work well with YOLOv4 in this task. Finally tracked vehicles’ traveled distance is back-projected to the 3D world by Haversine method for speed estimation. In order to deploy to edge device, we take the advantage of tensorRT framework and ONXX technology to optimize models and modify model format as well as accelerate inferencing. For the suitability to the Vietnamese traffic scenario, feature extraction models of tracking task and detection task were fine-tuned. The system is then optimized and modified to detect common Vietnamese vehicles’ speed in normal conditions and low light intensity conditions with the video stream fed directly from the preinstalled traffic surveillance camera. The whole system proceeds AI inference and processing with the help of Nvidia Jetson NX Xavier. All modules are packed into a small box which helps simplify the integration to available traffic cameras. This would release the need for radar and sensors which are usually extremely expensive and need a lot of calibration and maintenance.