Play With Me: Towards Explaining the Benefits of Autocurriculum Training of Learning Agents

Deep Reinforcement Learning (DRL) has firmly established itself in the smart grid domain in numerous applications, such as voltage control, real power distribution, or energy markets. Yet, many results presented suffer from an out-of-distribution sampling problem. This limits the general applicability of results as the agents can not be meaningfully judged on their performance in more realistic or adverse conditions. Autocurriculum training settings are assumed to remedy this, since the DRL-based adversary agent forces its counterpart to a more comprehensive sampling of the available state/action space and, subsequently, to develop a more comprehensive policy. However, until now, this assumption can only be inferred from the agent’s reward curve and other metrics stemming from the simulated environment. In this paper, we present an approach that uses an equivalent representation of an DRL agent’s policy network, which we then analyze to show the effects of autocurriculum training training. We also present an initial case study to demonstrate the our approach.