AutoReward: Closed-Loop Reward Design with Large Language Models for Autonomous Driving

Autonomous driving technology has made significant strides, with reinforcement learning (RL) proving crucial due to its superior decision-making capabilities. However, designing effective reward functions for RL in autonomous driving remains challenging due to the limitations of manual design, generalization issues, multidimensional complexity, and difficulties in quantification. This paper introduces AutoReward, a novel closed-loop framework that leverages large language models (LLMs) to automatically generate and optimize reward functions tailored for autonomous driving tasks. In this method, we guide the LLM to create reward functions from input task and environment code, and then iteratively refine these designs based on feedback collected from training the RL agent. The framework integrates Chain-of-Thought-based Environment Code Analysis and Task Requirements Decomposition modules to enhance the usability and effectiveness of the generated reward functions. Our approach reduces the complexity and time costs associated with manual design and demonstrates strong generalization across diverse traffic scenarios, including highways, intersections, and roundabouts. Extensive experiments show that our method improves key performance indicators, such as success rate and average success steps, by 15% to 30% compared to human-designed reward functions. Additionally, the framework exhibits broad adaptability with various LLMs and RL algorithms, marking a significant advancement in autonomous driving.