Study and Generalization on an End-to-End Spatial-temporal Driving Model

Autonomous driving has attracted increasing attentions from both academia and industry, among which end-to-end driving model is a nascent research topic. In this work a deep learning based model is studied and generalized, which directly maps raw videos to steering angles without any manual feature extraction procedure. The model is learned and evaluated from human driving videos from both public dataset and self-recorded dataset under real environment, which is more practical and application oriented. Besides, the model combines both the spatial and temporal information by applying 3D convolutional neural network and recurrent unit. For the dataset studied, ablation experiment is performed to further verify the unity of recurrent block. Finally, transfer learning is applied to transfer the domain knowledge learned from the large-scale public dataset to two different small-scale dataset in unseen scenes. Comprehensive quantitative and qualitative evaluations demonstrate that the RMSE on pretrained model is 0.12603, while the transfer learning for the two unseen scenes achieve 0.13079 and 0.13198, respectively, which shows the effectiveness and generalization ability of the model even under different scenarios and abrupt turns.