Solving an Artificial Mining Problem through Q-Learning

This paper details the process in which Q-learning was used to solve an artificial mining problem. Q-learning is a widespread and popular algorithm in reinforcement learning that returns an optimal policy for an agent to carry out in Markovian domains. It can either be used as a minimum optimization algorithm, where the cost is minimized, or as a maximization algorithm, where the reward is maximized. The mining problem is one we have created ourselves, and includes a $10\times 10$ map, various states determined by differently colored pixels, a single agent, and one single goal. Essentially, there are 6 gold mines in the map, and the goal is to collect all 6 gold mines with a minimized cost. The cost comes from each pixel that isn't a gold mine, and it varies between two different types of pixels. Furthermore, each gold mine is assigned a transition probability, which is the probability of either attaining the reward (for a gold mine) or receiving the cost (for non-gold mine pixels). This probability value makes our problem an indeterministic planning problem. We've implemented Q-learning to this problem through the programming language Python, and was returned a policy for the agent to take that allegedly maximized our reward from the gold mines. Our work is similar in some aspects to the SSP MDP model, but differs in that our transition probabilities reside within the states themselves, while the SSP MDP model is comprised of transition probabilities between the states. Therefore, through this paper, we've adapted the SSP MDP model into an unconventional application of Q-learning to solve our artificial mining problem.