Leader-follower Formation Control for a Multi-missile System via Deep Reinforcement Learning

The methods for the formation tracking control problem are mainly based on nonlinear control theories. Usually, the control effect of such methods heavily depends on an accurate control model and a lot of controller parameters need to be selected properly. To cope with these shortcomings, a novel reinforcement learning-based formation controller is proposed for a leader-follower multi-missile system. The deep deterministic policy gradient (DDPG) algorithm is implemented to generate the formation control signals. An interaction environment containing a leader and a follower is formulated, in which a model-free optimal control policy is obtained by a training process. The convergent actor-network is tested for a multi-missile system. Simulation results show that the formation control objective can be rapidly achieved in spite of the change of leader maneuvers. There is no need to change the hyperparameters of the policy for different kinds of formation scenarios, which shows great advantages over the conventional nonlinear control theory-based methods in terms of parameter selection.