Applying Reinforcement Learning and Supervised Learning Techniques to Play Hearthstone

Hearthstone is the most popular digital trading card game nowadays. This paper addresses the problem of applying machine learning methods for the development of an agent, which plays Hearthstone. We show how to separate a large action space of Hearthstone in two parts, one of which is quite small, in order to apply reinforcement learning. Also we develop a new tree-walking algorithm which allows us to consider only the most promising game states of Hearthstone in order to determine the best action sequence on a given turn. Besides that, we propose a new way to train and build a game state metric using supervised learning techniques. We develop four agents based on these proposals. Two of them show better results compared to previous works. Lastly, we show that the best developed agent can compete with game experts.