Learning Quadruped Locomotion Method Integrating Meta-Learning and Reinforcement Learning

The capacity for learning stands as a characteristic of advanced animal cognition. To explore the learning mechanisms underlying quadrupedal locomotive skills, this study delves into the gait learning task of quadrupedal robots, also reviewing several seminal algorithms proposed in recent years within the realm of reinforcement learning tailored to quadrupedal robotic platforms. In contemporary times, serving as an archetype in the domain of deep reinforcement learning, the Proximal Policy Optimization (PPO) algorithm has been extensively deployed in the context of quadrupedal gait learning tasks, yielding commendable experimental outcomes with a reduced demand for hyperparameters. However, the exclusive application of the PPO algorithm imparts a limitation on the agent's capacity to process high-dimensional information effectively. In response to the aforementioned concerns, sparked by insights from meta-learning and capitalizing on meta-learning's prowess in capturing high-dimensional abstract representations of the learning process, this paper introduces a novel fusion of meta-learning and the PPO algorithm, coined as the MAML-PPO algorithm. This innovation empowers quadrupedal robots to acquire enhanced gait proficiency. Simulated outcomes on the PyBullet simulation platform affirm that the algorithm posited in this study successfully imparts walking skills to quadrupedal robots. In comparison to the state-of-the-art SAC and PPO algorithms, the MAML-PPO algorithm advanced in this study not only accelerates training time but also bestows advantages such as heightened walking velocity upon quadrupedal robots.