Automatic Traffic Monitoring using Spatial Transformer Network and Monocular Depth Estimation

Automatic license plate recognition and vehicle speed measurement are two of the critical tasks in traffic monitoring that face several challenges. Existing CNN and RNN-based license plate recognition systems must manage significantly distorted input images or possess complex architecture leading to slower training and inference times. Speed measurement performed with dedicated hardware like various lasers, and Doppler radars have significant acquisition costs, while existing single camera solutions that use the geometric information of the road or homographic mapping have often cumbersome calibration methods and may not always provide accurate results. This paper proposes a license plate recognition solution based on a fully convolutional deep neural network combined with a spatial transformer network, which enables segmentation-free recognition of license plates while addressing the spatial weaknesses of convolutional networks. Additionally, a monocular camera-based speed measurement solution is proposed that builds on metric depth estimation algorithms and uses 3D reconstruction to estimate vehicle speed. The license plate recognition results show competitive accuracy and inference speed compared to other existing solutions while the proposed speed estimation method shows promising potential as an easy to calibrate and cost-effective solution.