Parallel Monte Carlo Tree Search from Multi-core to Many-core Processors

In recent years there has been much interest in the MCTS algorithm, a new, adaptive, randomized optimization algorithm. In fields as diverse as Artificial Intelligence, Operations Research, and High Energy Physics, research has established that MCTS can find good solutions without domain dependent heuristics. However, practice shows that reaching high performance on large parallel machines is not so successful as expected. So far, the reasons are not well understood. This paper investigates the scalability of two popular parallelization approaches (tree parallelization and root parallelization) of the MCTS algorithm, using the Intel Xeon Phi highly multi-threaded shared-memory system. Moreover, we compare the results on a Xeon CPU and a Xeon Phi to understand the scalability of the parallel MCTS algorithms, and to understand their absolute performance. We find that tree parallelization can achieve near perfect speedup for up to 16 threads on the Xeon CPU and up to 64 threads on the Xeon Phi. For root parallelization we find that the effect of locks is small. Moreover, we establish the overall parallel speedup of the two parallelization methods of the MCTS algorithm is fundamentally limited on the Xeon Phi for games such as Hex or Go. The limiting factor is not, as might be expected, the parallel algorithm, or its implementation, but the high level of sequential calculations in each thread, for which no vectorization method is known.