Inverse Reinforcement Learning with Failed Demonstrations towards Stable Driving Behavior Modeling

Driving behavior modeling is crucial in autonomous driving systems for preventing traffic accidents. Inverse reinforcement learning (IRL) allows autonomous agents to learn complicated behaviors from expert demonstrations. Similar to how humans learn by trial and error, failed demonstrations can help an agent avoid failures. However, expert and failed demonstrations generally have some common behaviors, which could cause instability in an IRL model. To improve the stability, this work proposes a novel method that introduces time-series labeling for the optimization of IRL to help distinguish the behaviors in demonstrations. Experimental results in a simulated driving environment show that the proposed method converged faster than and outperformed other baseline methods. The results also show consistency for various data balances of the number of expert and failed demonstrations.