Predicting Steering Actions for Self-Driving Cars Through Deep Learning

We propose a visual-based end to end lane following system which fuses temporal and spatial visual information to predict current and future control variables. Previous works only predict control variables for the next time point with the current visual information. In contrast, based on a long-term recurrent convolutional neural network, we investigate the effect of fusing history information of different lengths to predict the imminent control variable in different future horizons. Experimental results show that with long history visual information, the neural network can approximate human driving behaviours with high precision. Consistent with intuition is that the influence of history information declines as time moves forward. Meanwhile, history information of the past 0.6 seconds is of most information for the prediction, and the Mean Square Error (MSE) for the steering command prediction with 0.6s history information is 8.378 × 10
^-3
m
^-1
. By training the model with control signals that lag behind visual information as targets, the testing result shows that it is possible to predict future control variables with great accuracy, while the best prediction accuracy happens to the steering command 0.4 seconds later.