Deep Reinforcement Learning Based Control of Rotation Floating Space Robots for Proximity Operations in PyBullet

This paper presents a model-free learning-based controller for whole-body control of an orbiting space robot during proximity operations. Proximity control methods for space robots (robotic manipulators mounted on a floating satellite base) enable the robotic arm to reach up to the target body in order to perform autonomous tasks like in-orbit servicing, debris capture, etc. However, coupled motion control of such robots is tricky due to the floating nature of the satellite base. Although conventional controllers have been employed for coupled control of such nonlinear systems, their modeling and sophisticated control become all the more difficult with the increasing degree of freedom of the robot. Model-free Deep Reinforcement Learning (RL) has been successful in learning complex policies in the field of robotic manipulation. However, most of the research in this domain has been focused only on the control of the space robotic arm, and not the satellite base, with a majority of them focusing only on the arm position control. A coupled controller which also simultaneously controls the satellite base orientation is essential for the proper functioning of onboard sensors and equipment which have pointing requirements. This paper uses Proximal Policy Optimization (PPO) algorithm to control the position (3 DOF) and orientation (3 DOF) of the end-effector while also controlling the orientation of the base satellite (3 DOF). To the best knowledge of the authors, a model-free Deep RL method has not yet been used for simultaneous 9 DOF control of a floating space robot so far. We also improvise over the standard reward functions that are used in Deep RL algorithms for improved performance of the learning algorithm. The training of the policy is performed using a PyBullet physics simulator and the comparison of the performance of the learning algorithm against the standard reward functions is presented.