Matching an opponent's performance in a real-time, dynamic environment

In this paper, we explore high-level, strategic learning in a real-time environment. Our long-term goal is to create a computer game that provides a continuous challenge without ever being too difficult that discourages players or too easy that it bores players. Towards this goal, we propose an agent that is able to observe its environment, measure its performance against the human player(s), and carries out appropriate actions to maintain that challenge. The agent also learns about its reasoning process through reinforcement. We have applied our methodology to the video game Unreal Tournament 2003. The preliminary results are encouraging.