Imitation Learning for Autonomous Driving Based on Convolutional and Recurrent Neural Networks

Deep learning has pushed autonomous driving evolution from laboratory development to real world deployment. Since end-to-end imitation learning showed great potential for autonomous driving, research has concentrated on the use of end-to-end deep learning to control vehicles based on observed images. This paper utilizes convolutional neural network (CNN) together with recurrent neural network (RNN) so that the temporal information of the images can be used for the end-to-end deep learning control. Traditional RNN and two advanced variants, gated recurrent unit (GRU) and long short-term memory (LSTM), are separately applied in the end-to-end learning system. The vehicle speed and control signals computed from the trained models are shown to compare the three algorithms, in contrast to the original CNN-based model. It is demonstrated that with the same training parameters, the trained model with GRU gives remarkable improvement in the prediction accuracy for the speed, the steering angle and the throttle.