Deep Q-Learning for Aggregator Price Design

This paper develops a reinforcement learning (RL) aggregator agent that intelligently determines real-time electricity prices to achieve grid-level goals. A diverse community of home energy management (HEMS) systems, equipped with model predictive control (MPC) capabilities, respond to the prices without revealing their state, resources, constraints, or objectives to the aggregator. Only the aggregate response of the community is visible to the RL agent, preserving the privacy of each homeowner. The results indicate that the RL agent is successful in improving the flattening the aggregate community demand while adhering to individual user constraints without requiring two-way negotiations, transactive frameworks, or any knowledge about individual homes.