Motion Learning and Rapid Generalization for Musculoskeletal Systems Based on Recurrent Neural Network Modulated by Initial States

Musculoskeletal robot with high precision and robustness is a promising direction for the next generation of robots. However, motion learning and rapid generalization of complex musculoskeletal systems are still challenging. Therefore, inspired by the movement preparation mechanism of the motor cortex, this article proposes a motion learning framework based on the recurrent neural network (RNN) modulated by initial states. First, two RNNs are introduced as a preparation network and an execution network to generate initial states of the execution network and time-varying motor commands of movement, respectively. The preparation network is trained by a reward-modulated learning rule, and the execution network is fixed. With the modulation of initial states, initial states can be explicitly expressed as knowledge of movements. By dividing the preparation and execution of movements into two RNNs, the motion learning is accelerated to converge under the application of the node-perturbation method. Second, with the utilization of learned initial states, a rapid generalization method for new movement targets is proposed. Initial states of unlearned movements can be computed by searching for low-dimensional ones in latent space constructed by learned initial states and then transforming them into the whole neural space. The proposed framework is verified in simulation with a musculoskeletal model. The results indicate that the proposed motion learning framework can realize goal-oriented movements of the musculoskeletal system with high precision and significantly improve the generalization efficiency for new movements.

I. Introduction

Many application scenarios require robots to achieve high-precision and dexterous movements in fluctuating and unstructured environments. The human-like musculoskeletal robot is a promising way to satisfy these requirements and has received extensive attention. Compared with traditional rigid robots, the biological musculoskeletal system has obvious advantages. First, due to the redundancy of joints and muscles, it can achieve movements with high flexibility and robustness, even if a part of the actuators is fatigued or dysfunctional. Second, the stiffness can be modulated by coordinating the activation of agonist and antagonist muscles to adapt to different environments [1], [2]. Therefore, in order to demonstrate similar advantages, many musculoskeletal robots are designed with the imitation of the human musculoskeletal system in terms of muscular arrangement and driving mode [3]–[7]. Typical examples are the robots “Kengoro” built by the University of Tokyo and “ECCEROBOT” funded by the European Union’s Human Brain Project, these robots with human-like features, such as compliant, tendon-driven actuators, and complex joints exhibit high anatomical fidelity to the human musculoskeletal structure [7], [8]. In addition, some prosthetic limbs and exoskeleton robots inspired by the musculoskeletal system have also been developed, they have similar design principles and control strategies to musculoskeletal robots [9]–[12]. Dabiri et al. [9] have built an artificial prosthetic limb with antagonist artificial muscle structure, and it is driven by one kind of McKibben pneumatic muscle named Festo artificial muscle. Chen et al. [10] have implemented a 4-degree-of-freedom upper limb exoskeleton robot, which is actuated by pneumatic muscle actuators via steel cables.