Reinforcement Learning with Case-Based Heuristics for RoboCup Soccer Keepaway

In this paper we propose to combine Case-based Reasoning and Heuristically Accelerated Reinforcement Learning to speed up a Reinforcement Learning algorithm in a Transfer Learning problem. To do so, we propose a new algorithm called SARSA Accelerated by Transfer Learning - SATL, which uses Reinforcement Learning to learn how to perform one task, stores the policy for this problem as a case-base and then uses the learned case-base as heuristics to speed up the learning performance in a related, but different, task. A set of empirical evaluations were conducted in transferring the learning between two domains with multiple agents: an expanded version of Littman's simulated robot soccer and the RoboCup Soccer Keep away. A policy learned by one agent in the Littman's soccer is used to speed up the agent learning in the Keep away soccer. The results show that the use of this new algorithm can lead to a significant improvement in the performance of the learning agents.