Mixed Road User Trajectory Extraction From Moving Aerial Videos Based on Convolution Neural Network Detection

Vehicle trajectory data under mixed traffic conditions provides critical information for urban traffic flow modeling and analysis. Recently, the application of unmanned aerial vehicles (UAV) creates a potential of reducing traffic video collection cost and enhances flexibility at the spatial-temporal coverage, supporting trajectory extraction in diverse environments. However, accurate vehicle detection is a challenge due to facts such as small vehicle size and inconspicuous object features in UAV videos. In addition, camera motion in UAV videos hardens the trajectory construction procedure. This research aims at proposing a novel framework for accurate vehicle trajectory construction from UAV videos under mixed traffic conditions. Firstly, a Convolution Neural Network (CNN)-based detection algorithm, named You Only Look Once (YOLO) v3, is applied to detect vehicles globally. Then an image registration method based on Shi-Tomasi corner detection is applied for camera motion compensation. Trajectory construction methods are proposed to obtain accurate vehicle trajectories based on data correlation and trajectory compensation. At last, the ensemble empirical mode decomposition (EEMD) is applied for trajectory data denoising. Our framework is tested on three aerial videos taken by an UAV on urban roads with one including intersection. The extracted vehicle trajectories are compared with manual counts. The results show that the proposed framework achieves an average Recall of 91.91% for motor vehicles, 81.98% for non-motorized vehicles and 78.13% for pedestrians in three videos.