The role of reward structure, coordination mechanism and net return in the evolution of cooperation

Biology can offer insight into how realistic artificial agents and complex interactions between them can be created. For instance, spotted hyenas of Western Africa typically hunt alone, but cooperate once in a while to catch zebras. Using hyenas as motivation, this paper evaluates three potential factors that affect the evolution of cooperation in a team of game agents: reward structure (i.e. whether fitness reward is given to an individual agent or shared by the team), coordination mechanism (i.e. stigmergic vs. direct communication), and net return (i.e. the size of reward relative to the difficulty of obtaining it). Through neuroevolution using Multi-Component ESP [1], three predators were evolved in a field containing multiple fixed-behavior prey. Six such experiments show that (1) although shared reward strongly promotes cooperation, it is not strictly necessary for evolving cooperation; (2) although stigmergic coordination works well in simple, unambiguous tasks (i.e. with a single prey), direct communication is more effective in more complex tasks (i.e. with multiple prey); and (3) the predators evolve to hunt alone or cooperatively based on which approach results in higher net return. Insights from these computational simulations can be used to develop more intelligent behaviors for game agents in the future.