Simulation, learning, and optimization techniques in Watson's game strategies

The game of Jeopardy!™ features four types of strategic decision-making: 1) Daily Double wagering; 2) Final Jeopardy! wagering; 3) selecting the next square when in control of the board; and 4) deciding whether to attempt to answer, i.e., buzz in. Strategies that properly account for the game state and future event probabilities can yield a huge boost in overall winning chances, when compared with simple rule-of-thumb strategies. In this paper, we present an approach to developing and testing components to make said strategy decisions, founded upon development of reasonably faithful simulation models of the players and the Jeopardy! game environment. We describe machine learning and Monte Carlo methods used in simulations to optimize the respective strategy algorithms. Application of these methods yielded superhuman game strategies for IBM Watson™ that significantly enhanced its overall competitive record.