Multi-Spacecraft Fly-Around Control Method for Mixed Cooperative-Competitive Scenarios

Fly-around is important in many tasks such as on-orbit servicing. When a target exhibits game-playing behavior such as orbit maneuver for evasion, predesigned fly-around formation and control strategies can’t deal with the situation well enough. For mixed cooperative-competitive scenarios, this paper proposes a multi-spacecraft fly-around control method based on deep reinforcement learning. In order to motivate cooperative behavior among chasers, this paper designs structured reward function combining individual reward with collective reward for chasers. Meanwhile, in order to motivate competitive behavior of chasers, this paper designs appropriate reward function for target as practice partner. Self-play training mechanics using PPO algorithm is developed to train the chasers and the target simultaneously. Experiments show chasers can maintain appropriate formation around the target and adjust their formation to adjust to the changes of sunlight direction and unknown target behaviors.