Self-Evolutionary Reinforcement Learning for Autonomous and Robust Multi-Impulse Orbital Transfers

Autonomous, robust, and fuel-efficient trajectory optimization is critical for future spacecraft missions, enabling realtime orbital transfers under dynamic mission requirements. Nimpulse transfers between non-coplanar Earth orbits present significant computational challenges for traditional iterative methods, which rely on terminal conditions and lack adaptability to realtime state changes. To address this, we propose a Self-Evolutionary Reinforcement Learning (SERL) algorithm that integrates the evolutionary principles of competition and mutual learning from Particle Swarm Optimization (PSO) into the Reinforcement Learning (RL) framework. This integration enhances the algorithm's generalization and exploration efficiency. SERL divides the Nimpulse transfer process into N − 1 intermediate transition orbits, allowing the spacecraft to select intermediate orbits based on its observed states. Compared to traditional methods, SERL offers two key advantages. First, it generates transfer trajectories in real time once the target orbits are defined, enabling onboard deployment and improving spacecraft autonomy. Second, it bases decisions on immediate state observations rather than pre-defined terminal conditions, allowing it to handle complex tasks such as dynamically changing target orbits during mission execution. Numerical simulations validate SERL's ability to improve computational efficiency and adaptability for challenging orbital transfer scenarios