Obstacle-Avoidance Distributed Reinforcement Learning Optimal Control for Spacecraft Cluster Flight

The scientific goals of space missions place stringent performance demands on the cooperative control systems governing spacecraft clusters. Nevertheless, achieving the optimal distributed consensus control performance necessary to fulfill these requirements remains a persistent challenge, particularly in scenarios characterized by external disturbances and the imperative for collision avoidance implementation. Motivated by this fact, in this work, the problem of spacecraft cluster flight cooperative control with Reinforcement Learning (RL) and adaptive control with collision-free guarantees is addressed for rigid spacecraft cluster subject to system uncertainties and unexpected disturbances. First, the obstacle-avoidance performance functions specified long-term indexes by the designer are adopt to impose desired performance metrics on the collision-free behavior of spacecraft cluster. That is, the smaller long-term index is, the farther away from the obstacle, and the safer it is. Then, the collision avoidance conundrum faced by spacecraft undergoes a transformative shift towards an framework akin to the ”Min-max collision-free Actor-Critic mechanism of RL”. This framework effectively demonstrates the spacecraft's collision avoidance capabilities, sufficiently addressing the identified challenge by minimizing the maximum long-term indexes (Critic) within the spacecraft cluster through an optimized control strategy (actor). Subsequently, based on the optimal coordination scheme, a fully distributed control strategy is derived by incorporating the convex optimization, the adaptive control, the actor neural network (NN), the backstepping control, and the maximum consensus algorithm. It is proved that the designed optimal distributed cooperative controller is able to guarantee control targets without collision risk, as well as the boundedness of all closedloop signals, resorting to the Lyapunov stability theorem and convergence analysis of infinite series. Finally, the effectiveness of the proposed control protocol is demonstrated by exerting simulation experiments in spacecraft cluster flight.