Deep Reinforcement Learning for Autonomous Satellite Responsiveness to Observed Events

Autonomous and responsive space-based remote sensing is a sought-after and important need across the spacecraft-operations community. Building off several previous studies by other authors, this paper shows that an autonomous agent, trained using deep reinforcement learning (DRL), is capable of (a) autonomously managing the resources on board an Earth-observing satellite in low-Earth orbit, (b) autonomously executing collection on multiple prioritized ground targets, (c) autonomously downlinking collected data to a ground station, and (d) autonomously reacting, on-the-fly, to observed events without the assistance of human mission-operators on the ground. With respect to observed events, this paper is specifically concerned with on-orbit responsiveness to cloud cover. That is, if a ground target is seen to be occluded by clouds, the DRL-trained agent can autonomously transition the space vehicle to another, more fruitful, operational mode. This capability for on-orbit responsiveness is demonstrated here via three unique mission vignettes. The first shows the agent slewing to collect on a lower-priority target within the field of regard once a higher-priority target becomes occluded by clouds. The second mission vignette shows the agent deciding to switch to sun-pointing/standby mode when an ongoing collect on a ground target is suddenly obscured by cloud cover. The third vignette shows the transitioning to downlink mode when a nearby ground target becomes occluded. Taken together, these three mission vignettes demonstrate the versality of DRL-trained agent in being able to react, in real time, to observations made on orbit. The implications of such a capability are far-reaching and the paper closes with some thoughts on future directions for research in this area.