Assessing safe autonomous vehicle behavior via large scale traffic simulation

We study the problem of assessing and ranking the behavior of an autonomous vehicle using large scale traffic simulations with an emphasis on safety. We discuss a methodology to build a large-scale traffic simulation whose macro (or network) level properties are statistically close to real-world measured data. We use the Lankershim Boulevard data recorded as part of the FHWA-NGSIM initiative and build a traffic simulation in SUMO. We then assess the safety of an ego vehicle with different behavior models viz. Krauss, Intelligent Driver Model and the Extended Intelligent Driver Model. Several metrics such as Responsibility Sensitive Safety and Time-to-Collision are used to quantify the safety of ego-traffic interactions. With the proposed approach we aim to assess the behaviour of different autonomous vehicle driving stacks as part of internal regression testing and augment current scenario-based validation processes in simulation.