Context-Aware Gaussian Process for Physics-Informed Reinforcement Learning With Heterogeneous Data

Reinforcement learning (RL) with physics-informed dynamics models leverages known environments’ physics to improve data efficiency. Recently, we have introduced Gaussian Process Co-Adjustment (GPCA), a data-efficient model that conservatively refines simulator predictions, leading to the success of RL-controlled functional electrical stimulation (FES). However, GPCA’s behaviors and capabilities in broad settings remain unexplored. This study introduces context-aware GPCA (Ca-GPCA)—a novel extension of GPCA that incorporates a context encoder that learns latent representations (contexts) of transition histories. With context-augmented input, Ca-GPCA can generalize over heterogeneous datasets. Ca-GPCA consistently achieves 30% and 19% improvements in performance over GPCA in benchmark and FES control tasks, respectively, where the training data are collected from environments with different dynamics properties. These results demonstrate Ca-GPCA’s potential as a backbone for control systems that can operate instantly across diverse users and settings. In FES applications, for example, Ca-GPCA can utilize prior data from multiple individuals to perform in new users without requiring personalized calibration, paving the way for scalable, user-independent assistive technologies.