Distribution Grid planning by self-designing with reinforcement learning

Decentral energy resources are identified as indispensable for a successful energy transition with the target to reduce greenhouse gas emissions. This leads to amplifying electrification thereby posing new challenges to the distribution grid. In Germany, as in most countries, the electrical grid was designed based on central energy generation principles along with rather static load profiles. Given the two trends of electrification and decentralization, the distribution grid needs to be reinforced more dynamically than before. In this paper, a fully automated pipeline of grid planning is presented.Its central part is a novel (Multi-) Reinforcement Learning scheme operating on the electrical graph. It performs sequential grid planning actions such as switching, cable reinforcement and cable construction. Costs for construction are assigned by a cable route planner based on open street maps. Rewards depending on costs and load flow calculations on real grid data steer the optimization. It is shown that the optimizer develops strategies to solve grid violations while reducing associated costs for the actions. The framework is used to extract grid action sequences which are needed to mitigate increasing EV penetration. Furthermore, the dependency of EV density and associated grid action costs is evaluated quantitatively by running the optimizer for several scenarios. It is shown that costs increase rapidly when exceeding certain limit EV concentrations.