Understanding goals in learning by interaction

In reinforcement learning (RL) the goal of an agent is to maximize the sum of rewards that it receives. Agent designers must translate the desired goal into a particular reinforcement function. This process of translation is inherently error-prone because it is performed manually by human experimenters guided only by their experience and certain heuristic rules. It is a well-known phenomenon in RL when an agent rinds a way of maximizing the return without actually reaching the intended goal - all because of an incorrectly specified reinforcement function. For example, if in a game of chess we reward taking of the opponent's piece, the agent might find it more profitable to take as many as possible at the expense of loosing the game. In this paper we first examine the notion of a goal and then based on our understanding of goals propose a generalized way of imparting goal information to agents as an alternative to reinforcements. We show that using this approach we can significantly simplify goal specification by making it less prone to errors and in many cases reduce the memory requirements of learning algorithms. Preliminary experimental results with modified Q-learning algorithms are also reported.