Reinforcement Learning for Reconfigurable Robotic Soccer

Robots are showing great impact and, in recent trends, appearing in areas such as education and entertainment. Robotic soccer is becoming more prevalent in competitions, furthering research in robotics and artificial intelligence. Reconfigurable robotics is used in application domains such as cleaning, multi-terrain locomotion, and logistical support, but reconfigurability has yet to be introduced in robotic soccer. Using reconfigurable robots provides increased flexibility and adaptability in the game of soccer. This paper proposes Reinforcement Learning (RL) to train an agent to kick a ball toward a goal using reconfiguration. RL was used with the Proximal Policy Optimisation (PPO) algorithm to train and optimise goal scoring. The environment was developed and trained in Unity. Training included the agent learning to approach the ball in an optimal position to hit the ball into a goal using reconfiguration. Two use cases of penalty and free kicks were used to validate the accuracy of the proposed model, which resulted in goal conversion of 81% and 67%, respectively. Moreover, the results confirm that this method allows a reconfigurable robot to adapt to the soccer field and perform the best move out of the myriad possibilities in this complex yet competitive game.