A Simplified 2D-Model for Satellite Cooling Control Based on Deep Reinforcement Learning

Deep Reinforcement Learning (DRL) is an emerging control technique to solve optimal control problems. In this work, we develop a data-driven control approach that uses the DRL technique to intelligently design an effective strategy for operating the satellite cooling system. First, we construct a two-dimensional dynamic thermal environment for satellite panels by numerical simulation. Secondly, we employ an Artificial Neural Network, trained via a DRL agent, to execute satellite thermal control. Subsequently, we assess the agent's performance under extreme temperature conditions. Experimental results demonstrate that our method can effectively reduce the cumulative extreme temperature in simulation environments by 28%, 25%, and 13% respectively, compared to the traditional threshold-based approach across multiple environmental scenarios. Furthermore, the proposed method demonstrates robust performance across diverse noise conditions, yielding satisfactory results. Extensive experiments indicate that the model trained with TD3, which incorporates specific noise into sensors, displays significant robustness under high noise conditions.