A Study on the Effects of ICL and CoT Prompts on the Reward Function Generation of a Small Language Model

This study explored methods to improve the success rate of reward function generation, a crucial task in robotic reinforcement learning, by applying In-Context Learning (ICL) and Chain-of-Thought (CoT) prompting techniques to a Small Language Model (SLM). While the existing reward function generation framework, Eureka, used OpenAI's Large Language Model, GPT-4, this study focused on generating reward functions using the LLaMA3 8B model, which can be implemented on an edge device. The experimental results showed that the CoT technique alone was insufficient to enhance the performance. However, partial improvements were observed when the ICL technique was employed. Notably, combining ICL with CoT prompting techniques consistently maintained the success rate of reward function generation in both the cartpole and ball balance tasks. This could be interpreted as the complementary effects of the two techniques, where the ICL technique provided the model with context related to reward function generation and the CoT technique facilitated the solution of complex problems through step-by-step reasoning. Therefore, it was concluded that integrating various prompting techniques was crucial to improving the performance of SLM -based reward function generation. Future research should focus on increasing the experimental samples to derive statistically significant results; refining the combination of ICL generalization methods with CoT to reduce errors; and applying various prompting techniques to find the optimal strategy.