Reinforcement Learning for Optimal Stabilization of Magnetically Suspended Balance Beam Subject to Input Delay

In this paper, we present a delay compensating Q-learning algorithm for solving the optimal stabilization problem of a magnetically suspended balance beam system. The suspension by electromagnets mimics the operation of active magnetic bearings (AMBs) employed in industrial applications. We focus on the frequently encountered delay arising in the input current channel, which controls the magnetic field to produce the required force. We employ the recently proposed model-free reinforcement learning algorithm based on the idea of state augmentation to bring the balance beam system into a delay-free form. Compared to the recent works, the approach requires neither the knowledge of the system parameters nor the actual value of the delay. Only an upper bound of the delay is needed, which can be selected arbitrarily large as long as the learning time is acceptable. The stabilization of the balance beam and the convergence of the optimal control parameters are shown, confirming the efficacy of the proposed technique.