Spatio-Temporal Graph Policy Gradients for Multi-Robot Formation Control

Formation control is crucial for multi-robot collaboration. The high dynamics of multi-robot environments make it challenging to learn interactions between robots. Although some progresses have been made, the existing methods cannot capture spatial and temporal dependencies. In this paper, we propose a novel spatio-temporal graph policy gradients (STGPG), to tackle the formation control problem in the multi-robot domain. We propose using spatio-temporal graph convolutional network (STGCN) to parametrize the formation policy. STGCN adapts to the dynamics of the underlying graph in the formation environment, and uses the pure convolution blocks to capture the overall spatio-temporal correlations for learning formation strategies. The gradient of the robot not only back-propagates to itself, but also to its neighbors to reinforce the learned formation policy. Experimental results indicate the effectiveness of the proposed STGPG.