Improvement of Basic Behaviors for Bipedal Robot Based on PPO Algorithm

Deep Reinforcement Learning (DRL) makes a big improvement for robotic control. However, training stable and efficient bipedal gaits like walking and running still remains challenging. Current DRL approaches can be sample-inefficient and struggle with stability, while traditional methods may lack peak performance and adaptability. This paper proposes a novel approach integrating Proximal Policy Optimization (PPO) with Skill-Set-Primitives (SS-primitives), which predefined biomechanically-inspired motion patterns capturing commonalities across locomotion tasks. By providing a structured baseline, SS-primitives allow PPO to focus on refining performance and adapting to specific goals. We introduce a multi-objective reward function designed to prioritize key locomotion metrics. Experiments conducted in the SimSpark simulation environment demon-strate that our method significantly improves walking and running speed and stability compared to standard PPO training. The effectiveness of this approach was further validated by the Apollo3D team's success, achieving Champion at the RoboCup China Open 2024 and third place at RoboCup 2024 using this framework.