Q-learning based control algorithm with dynamic combination of peak shaving and self-consumption optimization for industrial battery storage systems

In the use of battery storage in industrial companies, the associated cost reduction depends strongly on the strategy used. Typically, companies use either self-consumption optimization or peak shaving as operation management strategies. In order to combine these two goals and achieve an optimal cost reduction, a control system must be developed that reacts dynamically depending on the current load peak of the prevailing month. For this reason, a Q-learning based control system was developed which reacts depending on the current peak load. In addition, a method was implemented to optimally design battery storage for this purpose in order to reduce investment costs and to conserve resources by selecting the smallest possible storage size. The reinforcement-learning agent was trained and validated on the basis of a real load profile of a local bakery chain. With a storage capacity of 27 kWh, a reduction of the peak load by 5.7 kW and an optimization of the self-consumption of 79% of the theoretical maximum could be achieved.