Fuzzy Action-Masked Reinforcement Learning Behavior Planning for Highly Automated Driving

Highly-automated driving relies on informed decision making and safe control in a dynamic environment which is regulated by traffic rules. This complex environment can be formulated as a map-based problem where the driving task is constrained by kinematics, obstacles, traffic, and road structure. Though, solving the problem with a monolithic approach additionally can lead to infeasible conditions which are difficult to bypass. Therefore, we expect a separate layer for sequential decision making to be a more suitable approach to derive a lateral and longitudinal behavior. In this area of decision making, reinforcement learning (RL) is a viable approach to complex and nonlinear problems. However, the learning convergence to a reasonable behavior in a critical and constrained environment can be slow. To overcome this challenge, we propose an action-masked RL agent utilizing fuzzy traffic rule descriptions. The agent outputs are hybrid lateral and longitudinal actions based on the environment observations. These driving decisions interface the subsequent trajectory planner and control for optimal execution. In exemplary merging scenarios, we show the effectiveness of the masked agent to increase convergence speed toward a reasonable behavior.