Policy Adaptation using an Online Regressing Network in a Soft Robotic Arm

This paper presents an error-driven adaptive scheme for soft arm control. It consists of two components: an embedded control policy and an online regressing network (ORN). The embedded control policy is trained to achieve a desired task in a training environment, and the ORN learns to adjust the input to the control policy to enable its implementation with a physical soft robot. The ORN accomplishes this by utilizing the mismatch information between the training environment and the physical soft robot. The control policy learns to follow a desired trajectory with appropriate accuracy in offline mode with a data-driven dynamics model of the soft robot using soft actor critic (SAC) algorithm. The trained policy upon testing with the actual soft robot exhibits significant training-to-reality gap. The proposed control scheme learns to overcome this training-to-reality gap in a matter of seconds. Its adaptability is further tested by employing a previously trained policy with the modified performance of the robot under constant external stress. It is observed that the control scheme manages to adapt to the new robot setting without needing to retrain the policy from scratch to achieve desired accuracy. This method could be particularly advantageous for developing a control solution that can be broadly applied to account for the stochastic behavior of soft robots, which may arise due to factors such as material hysteresis, manufacturing inconsistencies, inaccuracies in external tracking systems, and variable initial conditions.