Learning Traffic Behaviors by Extracting Vehicle Trajectories from Online Video Streams

To collect extensive data on realistic driving behavior, we propose a framework using online public traffic cam video streams. We implement the Traffic Camera Pipeline (TCP), a system that leverages recent advances in deep learning for object detection to extract trajectories from the video stream to corresponding locations in a bird's eye view traffic simulator. We benchmarked several deep learning detectors for the task of vehicle detection: SSD-VGGNet, SSD-InceptionNet, and SSD-MobileNet, and Faster-RCNN; we found that SSD-VGGNet had the highest precision and quality of bounding boxes. We captured four hours of video streams and used it to train generative models describing both the starting and ending positions of vehicles as well as the trajectories of points traversed. We find that the negative log-likelihood of held-out real data is more likely to occur under the learned models compared to baseline models used in a traffic intersection simulator. The extracted dataset of 234 annotated minute-long videos containing 2618 labeled vehicle trajectories and 8980 additional unannotated minute-long videos is available at https://berkeleyautomation.github.io/Traffic_Camera_Pipeline/.