Predicting unpredictable: Building models handling non-IID data, a Hearthstone case study

The following article is created as a result of the AAIA'17 Data Mining Challenge: Helping AI to Play Hearthstone. The Challenge goal was to correctly predict which bot would win a bot-vs-bot Hearthstone match based on what was known at the given time. Hearthstone is an online two-players card game with imperfect information (unlike chess and go, and like poker), where the goal of one player is to defeat their opponent by decreasing their “life points” to zero (while not allowing the opponent to do the same to oneself). Two main challenges were present: the transformation of hierarchically structured data into a two-dimensional matrix, and dealing with Non-IID data (certain cards were present only in test data). A way of how to successfully cope with those complications while using state-of-the-art machine learning algorithms (e.g. Microsoft's LightGBM) is presented.