Dynamic Programming for Creating Cooperative Behavior of Two Soccer Robots - Part 1: Computation of State-Action Map

To solve decision making problems of multi-agent systems, researchers have devised complicated methods, which are expected to solve curse of dimensionality. In this paper, we go to the opposite extreme. We generate cooperative behavior of two soccer robots with a simple dynamic programming (DP), which was proposed in '50s. Through the example, the ability of DP on a recent computer is measured and evaluated both qualitatively and quantitatively. We then show that the simple structure of DP is useful in obtaining behavior of robots in a convincing way. In the implementation of DP, space that is spanned by eight variables for decision making is divided into 610 million states. DP solves the optimal actions of two robots in every division and creates a look-up table, which is called a state-action map. The ability of this state-action map is measured by simulation and the result is discussed.