Learn collision-free self-driving skills at urban intersections with model-based reinforcement learning

Intersection is one of the most complex and accident-prone urban traffic scenarios for autonomous driving wherein making safe and computationally efficient decisions with high-density traffic flow is usually non-trivial. Current rule-based methods decompose the decision-making task into several serial sub-modules, resulting in long computation time at complex scenarios for on-board computing devices. In this paper, we formulate the decision-making and control problem under intersections as a process of optimal path selection and tracking, where the former selects a path with the best safety measure from a set generated only considering static information, while the latter then considers dynamic obstacles and solve a tracking problem with safety constraints using the chosen path. To avoid the heavy computation introduced by that, we develop a reinforcement learning algorithm called generalized exterior point (GEP) to find a neural network (NN) solution offline. It first constructs a multi-task problem involving all the candidate paths and transforms it into an unconstrained problem with a penalty on safety violations. Afterward, the approximate feasible optimal control policy is obtained by alternatively performing gradient descent and enlarging the penalty. As an exterior point type method, GEP permits control policy to violate inequality constraints during the iterations. To verify the effectiveness of our method, we carried out experiments both in simulation and in a real road test. Results demonstrate that the learned policy can realize collision-free driving under different traffic conditions while reducing the computation time by a large margin.