How to Run a Successful Game-Based AI Competition

Game-based competitions are commonly used within the Computational Intelligence (CI) and Artificial Intelligence (AI) in games community to benchmark algorithms and to attract new researchers. While many competitions have been organized based on different games, the success of these competitions is highly varied. This short paper is a self-help paper for competition organizers and aspiring competition organizers. After analyzing the fate of a number of recent competitions, some factors likely to contribute to the success or failure of a competition are laid out, and a set of concrete recommendations is offered. There is also a discussion of how to write up game-based AI competitions and what we can ultimately learn from them.