A Balance Control Method for Wheeled Bipedal Robot Based on Reinforcement Learning

Wheeled bipedal robot (WBR) fully combines the advantages of wheeled robot and legged robot, which enables it to adapt to more complex terrains and complete more challenging tasks. Therefore, WBR has been widely researched in recent years. However, at the same time, due to its under-actuation, nonlinearity, and instability characteristics, the balance control of WBR is a difficult problem. In this paper, a reinforcement learning (RL) algorithm based on path integral is proposed. First, the relationship between path integral and stochastic optimal control is analyzed, and the numerical solution of stochastic dynamical systems is obtained. Second, the solution framework is extended to a parameter update strategy of RL. Since the matrix inversion and gradient calculation are avoided during the strategy update process, the training is accelerated. Third, a model-based Lyapunov method is introduced to parameterize the control variables, which not only ensures the stability of the system in training, but also guarantees good performance of the system when applying simulation results to actual system. Simulation results show that the proposed method has satisfactory balance control performance and can effectively resist external disturbances.