Learning to understand multimodal rewards for human-robot-interaction using Hidden Markov Models and classical conditioning

We are proposing an approach to enable a robot to learn the speech, gesture and touch patterns, that its user employs for giving positive and negative reward. The learning procedure uses a combination of Hidden Markov Models and a mathematical model of classical conditioning. To facilitate learning, the robot and the user go through a training task where the goal is known, so that the robot can anticipate its user's commands and rewards. We outline the experimental framework and the training task and give details on the proposed learning method evaluating the applicability of classical conditioning for the task of learning user rewards given in one or more modalities, such as speech, gesture or physical interaction.