Modeling risk anticipation and defensive driving on residential roads with inverse reinforcement learning

There has been extensive research on active safety systems in the ITS community in recent years that has significantly contributed to reducing traffic accidents. However, further reduction is needed, especially on residential roads, where the reduction rate of traffic accidents is still quite small. On residential roads, traffic accidents are caused primarily by pedestrians suddenly running in front of cars and by the inattention of drivers to such risks. Automatic emergency braking systems activated by pedestrian detection are not always reliable on residential roads due to physical limitations such as too short a braking distance. To overcome the limitations of current active safety management systems, we focus on risk anticipation and defensive driving, key ideas to ensure safety on residential roads. Since defensive driving requires careful deceleration in advance of barrier lines and the corners of streets, long-term driver behavior prediction is needed. In this work, we provide a new framework of modeling risk anticipation and defensive driving with inverse reinforcement learning (IRL). In contrast to conventional driver behavior models such as hidden Markov models and maximum-entropy Markov models, our framework using IRL ensures accurate long-term prediction of driver maneuvers since the IRL is based on the Markov decision process (MDP), a goal-oriented path planning framework. Because the predicted defensive driver behaviors obtained by an MDP are appropriate only when the reward functions are carefully designed, we use inverse reinforcement learning, where the normative behavior of expert drivers is leveraged to optimize the reward functions. In addition to the proposed formulation of defensive driving with IRL, we provide new feature descriptors for computing reward functions to represent risk factors on residential roads such as corners, barrier lines, and speed limitations. Experimental results using actual driver maneuver data over 20 km of residential roads indicate that our approach is successful in terms of providing precise learning models of risk anticipation and defensive driving. We also found that the behavior models obtained by expert/inexperienced drivers are helpful for determining the factors in risk anticipation and defensive driving.