Social reinforcement in intrinsically motivated sensorimotor exploration for embodied agents with constraint awareness

This work introduces an intrinsically motivated exploration architecture which considers social reinforcement and motor constraint awareness. The architecture is used to study the modeling of early vocal development from a cognitive developmental perspective. Particular emphasis is placed on embodied cognition and sensorimotor exploratory behaviors. Experiments are performed for two systems, a simple toy example and a simulated vocal tract. In both cases, the existence of an expert instructor that reinforces the exploration with sensor units relevant to communication is assumed. The artificial agent is able to autonomously select goals to explore its own sensorimotor system in regions where a measure of competence to execute intended goals is improved. The agent is also endowed with a proprioceptive mechanism, based on a somatosensorimotor system, that is used to prevent the execution of unreachable motor configurations or invalid configurations. Additionally, during the exploration the instructor observes the sensor productions of the learner. When any sensor production is close to a speech utterance relevant to communication purposes then the instructor answers with that sensor unit. Immediately the learner attempts to imitate the instructor production. We argue that proprioception provides a way to deal with redundancy in the sensorimotor model and that social reinforcement provides clues to the learner of sensorimotor regions where it might be interesting to explore. The experimental results show that using both mechanisms, proprioception and social reinforcement, improves the performance of intrinsically motivated exploration architectures.