A Markov decision process approach to multi-timescale scheduling and pricing in smart grids with integrated wind generation

In this study, we tackle the challenge of integrating volatile wind generation into the bulk power systems, by lever-aging multi-timescale scheduling and pricing with two classes of energy users - traditional energy users and opportunistic energy users (e.g., electric vehicles or smart appliances). In day-ahead scheduling, with the distributional information of wind generation and energy demands, decisions on the optimal procurement of conventional energy supply and the day-ahead retail price are made; in real-time scheduling, with the realization of wind generation, the load of traditional energy users, the real-time prices are announced to manage the demand of opportunistic energy users so as to achieve system-wide reliability. Focusing on the case when the opportunistic energy users are persistent, i.e., they stay in the system until a real-time retail price is acceptable, we formulate the scheduling problem as a multi-timescale Markov decision process with special characteristics. We then show that it can be recast, explicitly, as a classic Markov decision process with continuous state and action spaces, the solution to which can be found via standard techniques.