Efficient Evaluation in Multi-Agent Reinforcement Learning via Rotation Invariance

Symmetry presents a prevalent challenge in multi-agent systems, holding significant research value for the compression of representation spaces. While remarkable progress has been made in studying permutation symmetry, research on rotation symmetry in multi-agent reinforcement learning remains relatively limited. Many multi-agent works still rely on discrete or artificially predefined finite transformation methods for rotation (duality) transformations, which are insufficient for generalizing policies in symmetric states. This paper presents a comprehensive investigation into the impact of rotational symmetry within Multi-Agent Reinforcement Learning (MARL) from an experimental perspective and analyzes the factors limiting its performance. To address the problem of state redundancy caused by rotational symmetry in MARL, we introduce an innovative approach that leverages state invariance to evaluate the $Q$ value. Through a series of experiments on SMACv2, the effect of rotation invariance on MARL is verified. Our proposed method and some existing methods can improve learning efficiency and convergence performance.