Latent Representation in Human–Robot Interaction With Explicit Consideration of Periodic Dynamics

This article presents a new data-driven framework for analyzing periodic physical human–robot interaction (pHRI) in latent state space. The model representing pHRI is critical for elaborating human understanding and/or robot control during pHRI. Recent advancements in deep learning technology would allow us to train such a model on a dataset collected from the actual pHRI. Our framework is based on a variational recurrent neural network (VRNN), which can process time-series data generated by a pHRI. This study modifies VRNN to explicitly integrate the latent dynamics from robot to human and to distinguish it from a human state estimate module. Furthermore, to analyze periodic motions, such as walking, we integrate VRNN with a new recurrent network based on reservoir computing (RC), which has random and fixed connections between numerous neurons. By boosting RC into a complex domain, periodic behavior can be represented as phase rotation in the complex domain without decaying the amplitude. A rope rotation/swinging experiment was used to validate the proposed framework. The proposed framework, trained on the collected experiment dataset, achieved the latent state space in which variation in periodic motions can be distinguished. The best prediction accuracy of the human observations and robot actions was obtained in such a well-distinguished space.