Royale Heroes: A Unique RTS Game Using Deep Reinforcement Learning-based Autonomous Movement

A game, a set of rules or policies by one or more people to carry out an action that will generate excitement for those who do it, can be developed using one of two types of playing methods: turn-based and real-time. In real-time strategic (RTS) games with many characters like Warcraft and Chess, it is fun to play. However, fun decreases since the players have to focus on many points in the playing area. Hence, autonomous movement (AM), an assistant to move the character, can be one solution. An AM can be implemented using deep reinforcement learning (DRL), a new method that can continue to learn without erasing previously learned memory that makes AM better since it can adapt to the playing style. This paper describes the Royale Heroes, a unique RTS game that implements a DRL-based AM. First, a base model is created using a rule-based model based on conventional artificial intelligence (AI). Next, a DRL model is trained to imitate the behavior of previously created AI. Evaluation of six matches shows that the DRL provides high accuracy of up to 98% to copy a behavior. Hence, it can be used as an AM to help players make choices in playing the game. When the player plays this game, the model will continue learning to imitate the player behavior.