Robot shaping: on learning and shaping in real robots

The mechanisms of learning, and the complex balance between what is learned and what is genetically determined, are one of the main concerns of behavioral sciences. It is our opinion that a similar situation arises in autonomous robotics, where a major problem is in deciding what should be explicitly designed and what should be left for the robot to learn from experience. We believe therefore that machine learning techniques are going to play a central role in the development of robotic agents. One important reason is that it might be very difficult, if possible at all, to incorporate enough world knowledge into an agent design from the very beginning. Presumably, agents can reach a high performance level only if they are capable of extracting useful information from their "experience", that is, from the history of their interaction with the environment. Robotic agents will have to organize their own behavior so that they can survive in their environment; but they will also have to perform some useful task. Therefore, agents' behavior will have to be properly "shaped" so that a predefined task is carried out: borrowing a term from experimental psychology, we call such a process robot shaping. We advocate an approach to robot shaping based on reinforcement learning.