Trajectory optimization of multistage launch vehicle’s Orbital Flight Phase based on reinforcement learning Algorithm

This paper studies the orbital flight phase trajectory optimization of multistage launch vehicle based on a proximal policy optimization(PPO)-long short term memory(LSTM) method that combined with a terminal step iteration(TSI) strategy. Traditional trajectory optimization algorithms have long computation times, making it challenging to meet the requirements of online planning. Trajectory optimization problems can be expressed by Markov decision processes(MDP), so they can be solved by reinforcement learning algorithms. PPO is an effective model-free reinforcement learning algorithm. However, the orbit tasks of launch vehicle has strong non-linearity and strict terminal constraints, which will make it difficult for the PPO algorithm to converge. Firstly, considering that the orbital flight phase is a sequential decision problem, LSTM network which can employ time series information is adopted to accelerate the rate of convergence. Then, this paper proposes the TSI strategy, which can significantly improve the terminal accuracy during the orbital injection process. Finally, by comparing the simulation results, the PPO-LSTM with TSI can generate guidance instructions online, and has better accuracy than traditional trajectory optimization algorithms.