Swarm Q-Learning With Knowledge Sharing Within Environments for Formation Control

A formation is a geometric shape that a group of agents spatially organizes themselves into and maintains over time. Swarm Q-Learning (SQL) is a tabular multi-agent reinforcement learning algorithm designed to solve formation control problems. We modify SQL by allowing agents to exchange knowledge they have learnt within the same environment and introduce the Swarm Q-Learning with knowledge Sharing wIthin an Environment (SQL-SIE). The algorithm is tested on a task where a swarm of robots, initially scattered in one side of the environment, needs to navigate through obstacles until they reach their initial positions in the formation within a region of interest. Experimental results show that the proposed SQL-SIE is more efficient than SQL as measured by the time taken by the swarm to complete this part of the mission. Moreover, SQL-SIE scales better than SQL as the number of agents increases.