Constructing competitive and cooperative agent behavior using coevolution

In nature, multiple agents in teams collaborate and compete with one another at the same time. Replicating such agent interactions in games can make for realistic opponent teams. Yet cooperation and competition have mostly been studied separately so far. This paper focuses on simultaneous cooperative and competitive coevolution in a complex predator-prey domain. Multi-Agent ESP [23] architecture is first used to evolve neural networks to control predator and prey agents, but such a naive combination of otherwise successful architectures turns out not to sustain an arms race. An extended architecture consisting of multiple cooperating neural networks within each agent is therefore introduced. This architecture successfully results in hierarchical cooperation and competition in teams of prey and predators: In sustained coevolution, high-level pursuit-evasion behaviors emerge. In this manner, coevolution of neural networks is shown to scale up to an arms race of multiple competing and cooperating agents, more closely modeling coevolution of complex behavior in nature.