Inverse Reinforcement Learning via Neural Network in Driver Behavior Modeling

Inverse Reinforcement Learning(IRL) is formulated within the framework of Markov decision process(MDP) where we are not explicitly given a reward function, but where instead we can observe an expert demonstrating the task that we want to learn to perform. Then the expert as trying to maximize a reward function that is expressible as a linear combination of known features specifying the reward function. However, in autonomous driving tasks, due to the difference of scene factor, such as obstacle and weather, the state spaces are frequently large and demonstrations can hardly visit all the states. it's hard to get an optimal policy with RL method to express driver behavior model based on the reward which recovered with IRL method in this large-scale state space. In this paper, we focus on driving behavior modeling with IRL method which introduces the convolutional neural network to extract the associated state feature automatically, and express the policy by neural network to generalize the expert's behaviors. Experimental results compared with the traditional end-to-end method on simulated vehicle show that the accuracy of decision-making greatly improved in the train curve, and in the new curve scene with a large number of unvisited state, this method shows a perfect generalization efficiency.