Localization in Urban Environment Using GNSS and Infrastructure-based Camera

Cooperative localization between vehicles and infrastructure is pivotal in overcoming bottlenecks of localization accuracy with respect to multipath and non-line-of-sight (NLOS) effects for urban operations. Within the vehicle-to-infrastructure (V2I) principle and intelligent transport system (ITS), roadside sensors are capable of providing additional measurement to enhance a vehicle's localization performance. This paper introduces a cooperative localization framework that uses infrastructure cameras from roadside unit (RSU) to aid in minimizing positioning errors for ego vehicles (EV s). Positions extracted from RSU cameras are loosely fused with local Global Navigation Satellite System (GNSS) information along with inertial measurement unit (IMU) readings using extended Kalman filter (EKF). To compensate for localization errors, Long Short-Term Memory (LSTM) is applied to augment fusion accuracy by learning geographic positioning error distributions with ground truth from the training dataset. By demonstrating the framework over CARLA, the experimental results indicate that under normal GNSS conditions, the LSTM-based fusion method reduces the RMSE of the position error by 38% compared to the EKF. After introducing GNSS bias and considering GNSS outage, the RMSE of the position error is reduced by 31 % and 30%, respectively.