Image Prediction for Lane Following Assist using Convolutional Neural Network-based U-Net

Current autonomous driving systems compute steering and throttle control commands by running perception-decision-action pipeline at high frequency. Although human drivers cannot react or control the vehicles as quickly as the autonomous driving softwares, most drivers control their vehicles to stay in lane unless they intend to break away from the lane. According to forward internal model theory, human can choose an optimal action for the best outcome by internally simulating all the possible consequences of various actions. This means that humans drivers choose the optimal motor commands for lane following based on their internal simulation of near-future lane changes. This paper proposes a convolutional neural network-based U-Net as a state estimator for forward internal model-based lane following assist. This state estimator can predict the lane image of near-future based on current lane image and driving status data, such as speed and steering angle. This paper also explains how time difference between current lane image and the next one to be predicted will affect the training and prediction output of the estimator.