Research on LLRF Feedback Control Algorithm Based on Reinforcement Learning

This study proposes a Low-Level Radio Frequency (LLRF) feedback control algorithm based on reinforcement learning using the soft actor-critic and proximal policy optimization algorithms. These approaches can automatically learn optimal control strategies for multi-input multi-output systems, thereby reducing interference among variables during closed-loop control and enhancing the overall control precision of the system. To avoid interruption to accelerators when training the reinforcement learning models, we modeled the relationship between LLRF operator input and klystron output to enable offline training, significantly accelerating the training. Trained reinforcement learning models were validated at the FELiChEM facility. Experimental results demonstrated that the LLRF control algorithm based on reinforcement learning shows improvements in both amplitude and phase closed-loop accuracy compared to the traditional univariate PI controller.