Constrained Thompson Sampling for Real-Time Electricity Pricing With Grid Reliability Constraints

We consider the problem of an electricity aggregator attempting to learn customers' electricity usage models while implementing a load shaping program by means of broadcasting dispatch signals in real-time. We adopt a multi-armed bandit problem formulation to account for the stochastic and unknown nature of customers' responses to dispatch signals. We propose a constrained Thompson sampling heuristic, Con-TS-RTP, as a solution to the load shaping problem of the electricity aggregator attempting to influence customers' usage to match various desired demand profiles (i.e., to reduce demand at peak hours, integrate more intermittent renewable generation, track a desired daily load profile, etc). The proposed Con-TS-RTP heuristic accounts for day-varying target load profiles (i.e., multiple target load profiles reflecting renewable forecasts and desired demand patterns) and takes into account the operational constraints of a distribution system to ensure that customers receive adequate service and to avoid potential grid failures. We provide a discussion on the regret bounds for our algorithm as well as a discussion on the operational reliability of the distribution system's constraints being upheld throughout the learning process.