Robot Episodic Memory to ‘Learn, Remember and Anticipate’ User Interactions for Proactive Assistance

For those working in care environments being able to interact and provide personalized assistance to a wide range of users with diverse needs is a fundamental requirement. For a robotic companion working within the same environment, the ability to distinguish users and predict behaviors/needs based on past experience is hence essential for successfully providing the correct assistance. This article proposes a growing neural episodic memory that enables a Pepper Humanoid to learn its interactions with different users. Such experiences stored in the episodic memory network can be recalled based on multimodal partial cues in the present (detected face, objects, actions like ‘user taking a medicine’ etc.) enabling the robot to anticipate future states and goals so as to provide proactive assistance. The episodic memory network itself is integrated to a range of subsystems related to perception and action to build an integrated architecture for providing customized assistance and social engagement. The framework is validated in two ways; First, by having the robot detect users taking performing harmful actions (such as taking incorrect medication) and correcting the behavior and secondly, by showing how knowledge learned from one users can be transferred and applied to a new user to provide assistance.