Energy-Based Exploration for Reinforcement Learning of Underactuated Mechanical Systems

Reinforcement Learning (RL) for underactuated mechanical systems presents unique challenges due to the limited control inputs and complex dynamics of the system. Efficient exploration poses a significant problem for two reasons: First, these systems must start each episode of learning from an equilibrium position instead of a random position, severely limiting exploration. Second, these systems are often chaotic, which limits model-based exploration. To address these challenges, we introduce Energy-based Exploration for Reinforcement Learning (EBERL), which exploits a model of the system’s energy represented by a Deep Lagrangian Network (DeLaN), coupled to an energy-based controller to ensure goal-directed exploration on top of the exploration principles that are already present in RL. First, we validated our method in simulation on a Cartpole, Furuta pendulum and Pendubot with challenging torque constraints that cannot be solved by traditional methods. Additionally, we experimentally validated our method on the swing-up and stabilization of a Quanser QUBE2 Furuta pendulum.