Reinforcement Learning for PID Gain Tuning in Ball-and-Beam Systems: A Comparative Study

In this paper, the use of Reinforcement Learning (RL) to fine-tune Proportional-Integral-Derivative (PID) controller gains for a ball-and-beam control system is explored. Rather than a simulated ball-and-beam system, we implement a low-cost system consisting of a 3D printed housing, a servo-mounted acrylic beam, a time-of-flight laser-ranging sensor, and a standard Ping-Pong ball. The PID controller serves as a baseline for control, and RL agents adjust the gains to optimize system performance depending on the angle of the beam, position and speed of the ball, and the current setpoint. Agents utilizing the reinforcement learning (RL) algorithms A2C, PPO, SAC, DDPG, and TD3, are trained and the controller performance are evaluated based on system response and disturbance rejection. Practical tests reveal that stable control was achieved by PPO, SAC, and TD3 agents within 6.356, 2.058, and 6.265 seconds, respectively, with the PPO and TD3 agents demonstrating the capability to recover from disturbances.