Hierarchical Multi-Agent Training Based on Reinforcement Learning

In the current multi-UAV adversarial games, issues exist such as the instability and difficulty in learning distributed strategies, as well as a lack of coordinated formation UAVs. In this paper, a hierarchical multi-agent training framework is proposed to solve these problems, which categorizes UAV formations into two types of intelligent agents: virtual centroid agents and UAVs within the formation. The centroid agents are responsible for controlling the overall movement of the formation. In contrast, the UAVs within the formation are capable of flexibly adjusting their speed and heading on this basis. By constructing a confrontation scenario involving multiple formations and types of UAVs, the effectiveness of the hierarchical training framework is experimentally validated. The average winning rate against UAVs controlled by strategy methods based on rule construction reaches ${9 7 \%}$, enabling both formation variations and tactical evolutions.