An Optimal Re-parametrization Scheme for Generalization in Reinforcement Learning

The use of Reinforcement learning (RL) in addressing a variety of engineering tasks has seen a lot of increase lately. However, because RL policies are trained under specific environmental conditions, their performance degrades when deployed in environments with different environmental conditions. As a result, several researchers have focused on developing schemes aimed at minimizing this performance degradation and consequently realizing RL agents that are capable of generalizing to different environmental conditions. In this paper, we propose an optimal re-parameterization scheme based on weighted averaging to facilitate generalization in RL. In the proposed method, two RL agents are trained on simulated environments with different environmental or model parameters, and then covariance matrix adaptation evolution strategies (CMA-ES) is used to determine the optimal averaging weights to combine the two agents for a test environment. We evaluate the performance of our method on a set of popular RL locomotion environments and show that it can significantly improve the generalization performance of RL policies.