Lane Change Detection with an Ensemble of Image-based and Video-based Deep Learning Models

Prediction of lane-changing maneuvers of surrounding vehicles is important for autonomous vehicles to understand the scene properly. This research proposes a vision-based technique that only requires a single in-car RGB camera. The surrounding vehicles’ maneuvers are classified as right/left lane-change or no lane change conforming to most lane change detection studies in the literature. The usual practice in previous studies is feeding individual video frames into CNN to extract features and afterward using an LSTM to classify the sequence of features. Differently, in our study, we exploit the power of ensembling the prediction results of two methods. The first one uses a small feature vector containing the image coordinates of the target vehicle and classifies it with an LSTM. The second method works with a simplified scene representation video (only the target vehicle and ego-lane highlighted) and it is based on a self-supervised contrastive video representation learning scheme. Since maneuver labeling is not required in the self-supervised learning step this enables the use of a relatively large dataset. After the self-supervised training, the model is fine-tuned with a labeled dataset. Our experimental study on a well-known lane change detection dataset reveals that both of the mentioned methods by themselves achieve state-of-the-art results and ensembling them increases the classification accuracy even more.