Reinforcement Learning in Energy Management: PV & Battery Storage for Consumption Reduction

Thailand’s steady increase in electricity costs has led to a rapid growth in solar rooftops and batteries installations for energy storage in households. These trends have presented an opportunity for the development of cost-efficient algorithms that optimize energy management. To accomplish this goal, we employed Reinforcement Learning (RL) to optimize energy management by regulating battery charge and discharge, while simultaneously reducing peak demand to mitigate demand charges associated with electricity consumption. We compared various state-of-the-art RL algorithms, including Advantage Actor-Critic (A2C), Augmented Random Search (ARS), Deep Q-Network (DQN), Proximal Policy Optimization (PPO), Quantile Regression DQN (QRDQN), Recurrent PPO (R-PPO), and Trust Region Policy Optimization (TRPO), to a baseline model referred to as Load FIRST. Load FIRST is a rule-based default algorithm commonly used in various solar inverter brands. Our study revealed the promising potential of RL algorithms to optimize battery power management for energy savings, specifically in the rapidly expanding solar rooftop and battery storage market in Thailand. The ARS model, in particular, yielded the most substantial reduction in electricity costs. The cost of electricity generated by the ARS model was 1,068.73 Baht, representing an 18.47% (217.65 Baht) lower than the baseline cost of 1,286.38 Baht. Our results suggested that employing RL algorithms for battery management optimization could reduce both peak demands and electricity costs.