Reinforcement Learning Enhanced Controller Synthesis for Autonomous Vehicles

Developing control programs for autonomous vehicles is a challenging task, mainly due to factors such as complex and dynamic environments, intricacy of tasks, and uncertain sensor information. To tackle the challenge, this paper harnesses the potential of formal methods and deep reinforcement learning (DRL) for a more comprehensive solution that integrates Generalized Reactivity(1) (GR(1)) synthesis with DRL. The GR(1) synthesis module takes care of high-level task planning, ensuring a vehicle follows a correct-by-construction and verifiable plan for its mission. On the other hand, the DRL model operates as the low-level motion controller, allowing the vehicle to learn from experience and adjust its actions based on real-time sensor feedback. Therefore, the resulting controller for autonomous vehicles is not only
guaranteed
to finish its designated tasks but also
intelligent
to handle complex environments. Through comparative experimental studies, we demonstrate that the control program generated by the proposed approach outperforms the ones generated independently utilizing GR(1) reactive synthesis and DRL.