Reward Design Framework Based on Reward Components and Large Language Models

The application of Large Language Models (LLMs) in the field of robotics has gained widespread attention. Due to their powerful code generation and contextual understanding capabilities, these models can generate reward functions from task commands, prompts, and environmental code, thus aiding robots in acquiring skills through reinforcement learning (RL). However, reward functions generated by existing models often suffer from issues of low executability and success rates, particularly when handling vague or complex task commands. These models usually require multiple iterations of optimization to achieve the desired outcomes. To address this challenge, we propose a reward function generation method based on reward components, which leverages human-level prior knowledge to improve generation efficiency and accuracy. Specifically, we have constructed a task→reward components dataset and fine-tuned a Reward Component Generator (RCG) using this dataset. The RCG then guides the automatic generation of reward functions for reinforcement learning tasks. Experimental results demonstrate that our method significantly improves reward executability, accuracy, and task success rate compared to state-of-the-art approaches. For more information, please visit our project website at: https://jkx-yy.github.io/RCG/