Peer-to-peer Electricity Transaction Decisions of the User-side Smart Energy System Based on the SARSA Reinforcement Learning

With the deep integration of advanced information technologies, such as artificial intelligence and traditional energy technologies, smart energy systems have been proposed as a method to provide the best solution for the coordination, balance, and control of the entire energy system. As a new way of energy balance and interaction in the user side energy market, a peer-to-peer (P2P) electricity transaction can effectively promote energy sharing within the user group and improve the economic benefits of users participating in the energy market. Reinforcement learning (RL) is an artificial intelligence method in which agents continuously acquire relevant experience and knowledge during the interaction with the environment, automatically update their decision-making behavior, and achieve maximum return. It is a suitable approach for P2P transaction decision analysis of small-scale users in the context of smart energy. First, this paper establishes a P2P transaction model that includes a participant model, equipment model and price model. Secondly, the transaction problem is equivalent to a Markov decision process (MDP) and each learning element model is established. Then, the MDP problem is solved and analyzed using the SARSA RL algorithm with average discrete processing. Finally, a case study of a community with multiple users is conducted to verify the effectiveness, economy, and security of the RL method in solving energy storage action selection and transaction decision problems of energy storage users.