A Learning-Based Vehicle Trajectory-Tracking Approach for Autonomous Vehicles With LiDAR Failure Under Various Lighting Conditions

Autonomous vehicles have been widely equipped with radars, camera, and LiDARs due to their complementary capabilities of environment perception. However, it becomes a critical task to accurately track the trajectory of the preceding vehicle when the host vehicle suffers a LiDAR failure under challenging lighting conditions. This article proposes an integrated learning-based solution to address this critical issue. It is composed of a
Q
-learning-based Gaussian mixture model (QLGMM) for clustering dense radar data, a weight-scheduled method for radar data association, and a switchable recurrent neural network with dual-level long short-term memory (LSTM) cells for trajectory tracking under LiDAR failure and various illumination levels. Specifically, the QLGMM is first introduced to improve the conventional GMM-EM algorithm with the cluster number determined by a
Q
-learning approach. Then, the weight-scheduled method is presented to associate the data from multiple radars. Furthermore, a switchable dual-level LSTM network is developed to adaptively fuse the trajectories from the radar and camera streams based on three lighting modes. The training data and testing data were acquired on a fully instrumented autonomous vehicle. Experimental verification demonstrates that the proposed method can achieve a promising improvement for LiDAR-fault-tolerant trajectory tracking under different lighting conditions.