Decision-making with Triple Density Awareness for Autonomous Driving using Deep Reinforcement Learning

Deep reinforcement learning (DRL) has been a recent trend to solve autonomous driving decision-making (DM) problems. The goals of DM are to produce actions with ensuring safety and improving efficiency according to the information of perception and localization. Intuitively, the complexity and density of the driving scene will affect the DM while few works focus on it. This work is the first to integrate density characteristics in DRL. Specifically, we model three density indicators, 1) local density to describe the vehicle density relationship around the ego vehicle, 2) lane density to describe the denseness of vehicles on each lane, 3) global density to describe the denseness of current region of interest of DM. Among them, local density and lane density are considered as input features to make the agent aware of density properties to distinguish dense scenes while global density is used as a factor of the reward function to balance the safety and efficiency. The agent learns to make full use of density information to help the current DM process through learning. We train and test the models in an environment with different density scenarios. The experimental results show that considering density in DRL significantly helps the agent learn to make more secure and efficient decisions for autonomous driving.