Deep Reinforcement Learning for Rendezvous and Attitude Control of CubeSat Class Satellite

Advancements in Reinforcement Learning (RL) algorithms and technologies have opened up new possibilities for their use in autonomous spacecraft control. This work presents two novel Deep Reinforcement Learning (DRL) agents which, when combined, can achieve satellite rendezvous while simultaneously maintaining attitude control. This control method allows for simultaneous, autonomous control that can appropriately actuate a spacecraft even in situations the agent has not previously encountered. The rendezvous controller consists of a Twin-Delayed Deep Deterministic Policy Gradient (TD3) agent that would be able to control a CubeSat and achieve rendezvous with the target satellite. The attitude controller consists of a quaternion-based TD3 agent; it has shown the ability to perform large angle slew manoeuvres and achieve pointing accuracies of less than one degree error while resisting environmental perturbations. The controller’s feasibility was assessed using a six-Degree-of-Freedom (6-DoF) relative spacecraft dynamics model, along with a model of the spacecraft’s environment. The agents were trained in a simplified environment and subsequently tested in an environment that incorporated disturbance models and actuator noise, providing an evaluation of their capabilities in realistic conditions.