Comparison of Reinforcement Learning Algorithms for Speed Ripple Reduction of Permanent Magnet Synchronous Motor

In this paper, a reinforcement learning based approach for reducing the speed ripple of a permanent magnet synchronous motor is presented. The method assumes that the speed ripple is caused by a sinusoidal disturbance with known frequency. A test bench with a mechanical load, which causes such a sinusoidal disturbance due to a static unbalance, is used to evaluate the method. Different reinforcement learning algorithms (Q-learning, double Q-learning, SARSA) are compared by the means of simulation. The simulation results are verified on the test bench and experimental results are presented. It is shown that it is possible to learn a feed forward compensation method by RL agents. However, the performance of the method has a high dependency on the random seeds in all three investigated RL algorithms and does not learn this result reliably. Further investigations are necessary to determine the cause and possible mitigations of this lack of robustness of the implemented algorithms to enable industrial applications of reinforcement learning.