Learning and Adapting Behavior of Autonomous Vehicles through Inverse Reinforcement Learning

The driving behavior of autonomous vehicles has a significant impact on safety for all traffic participants. Unlike current traffic participants, autonomous vehicles in the future will also need to adhere to safety standards and defined risk properties in order to achieve a high level of public acceptance. At the same time, successful autonomous vehicles must be able to interact with human drivers in mixed traffic in a way that enables traffic to flow. In this paper, we present a hybrid approach to trajectory planning that learns and adapts human driving behavior using inverse reinforcement learning. The proposed approach performs a large-scale simulation with HighD real-world scenarios to learn human driving behavior and domain-specific traffic-flow characteristics. The analysis of the work focuses on the influence of risk-taking, which provides information about driving style safety. The results show insights into the risk behavior of trajectory planning approaches compared to human risk assessment. The comparison to human trajectories is intended to ensure comparability and accurate classification of risk-taking. We recommend a hybrid method for adapting driving behavior, in order to maintain the explainability and safety of the trajectory planning algorithm.

I. Introduction

The reality of autonomous driving is drawing ever closer. The first autonomous vehicles (AVs) are proving themselves in mixed traffic situations involving human drivers and AVs. The behavior of AVs and the way they drive will be crucial to the success of the technology. In addition to subjective perception and comfort metrics, risk metrics will also have an important impact on road participants’ acceptance. However, some residual risk is required for the vehicle to reach the destination in an acceptable time [1]. In the near future, the risk of AVs in mixed traffic with human-controlled vehicles cannot be ruled out, as traffic flow must be maintained. To date, nearly all vehicles on public roads have been controlled by human drivers who take full responsibility for their actions. The fundamental reference point for a robust driving behavior is therefore still the human being. At the same time, it is clear that human driving has its limits, and human error is still the main cause of traffic accidents and fatalities. For acceptance, safety, and smart autonomous driving, it is worth looking at human driving behavior to identify differences between the theoretical optimal driving behavior and situational awareness of human drivers. In particular, an interesting aspect emerges when considering risk and risk distribution in road traffic, as society seems to accept higher risks for human drivers than for autonomous systems [2]. However, since we do not know how much country-specific risk for AVs will be accepted by society, we need to be able to adjust AV driving behavior according to future requirements. At the same time, inverse reinforcement learning (IRL) algorithms are becoming increasingly popular to learn driving behavior. To date, there have been no findings or results of an IRL-accelerated humanistic driving behavior approach combined with risk-aware planning. The results in this field are expected to shed light on human driving behavior and provide information for the societal discussion on how the AV of the future should behave. A study between different categories of driver types illustrating the range of risk tolerance of human drivers in different situations expands the research field. In this work, an examination of which level of risk is the right one in terms of ethical standards will not be considered. In summary, this work presents three main contributions: 1)

We present a hybrid analytic trajectory planning algorithm informed by an IRL model to adapt human driving behavior using recorded trajectory datasets. We illustrate the practicability, high safety, and transparency of our approach.

2)

We compare our approach to the risk-taking behavior of human drivers and a trajectory planning algorithm that applies risk metrics to improve the safety of all road users [3].

3)

We cluster real-world data according to different human driving style types, in order to classify and compare them with our approach in a traffic situation on a German highway at high speeds through a large-scale simulation. We use the CommonRoad environment [4] to show the optimal balancing act between risk tolerance and safety.