Foresighted Decisions for Inter-Vehicle Interactions: An Offline Reinforcement Learning Approach

Adaptive decision-making capability is essential for autonomous driving systems that require interaction with other drivers and is concerned with safeness and efficiency. Although several attempts have taken, e.g., model-based online reinforcement learning, the online learning process is impractical, and it is challenging to build a model reflecting a multi-agent environment. This study aims to create an autonomous driving agent who can make adaptive decisions using foresighted future information among other drivers. To endow this functionality into the agent, we introduce episodic future thinking (EFT) to a partially observable Markov decision process (POMDP) and mathematically define it. We adopt the offline learning paradigm, enabling an agent to leverage foresighted information preprocessed from the previously collected dataset in a training phase. Consequently, we benchmark the state-of-the-art offline reinforcement learning algorithm and compare the performance between vanilla and EFT-based reinforcement learning.