Variable Impedance Control Combining Reinforcement Learning and Gaussian Process Regression

Variable Impedance Control (VIC) approaches offer effective means for enabling robots to perform physical interaction tasks safely and proficiently, by including time-varying gains within an impedance control loop. However, determining the optimal gain profiles can be tedious and time-consuming. To address this challenge, this study introduces a VIC learning framework capable of autonomously acquiring suitable impedance behavior during task execution. This achievement is realized through the fusion of two techniques: (i) Reinforcement Learning (RL), to determine the most appropriate stiffness and damping gains for solving interaction tasks (e.g., lifting, pushing); and (ii) Gaussian Processes (GPs) for modeling and estimating optimal impedance parameters across task variations (e.g., changes in object weight). Consequently, we propose a Fast Cross-Entropy Method (FCEM) algorithm for autonomous stiffness learning, emphasizing all-the-time-stability to guarantee the stability of the control loop throughout the RL process. Additionally, we present a GP-based method to adapt impedance behaviors at run-time, adjusting stiffness based on online external torques estimates provided by a momentum observer (without requiring a wrench sensor). Experimental results on a simulated ABB Mobile YuMi robot show the framework’s capabilities across different tasks.