Dispatching thermostatically controlled loads for frequency regulation using adversarial multi-armed bandits

Utilizing residential Thermostatically Controlled Loads (TCLs) for demand response stands to offer a more economical and environmentally friendly alternative to procuring energy storage and generation facilities for grid ancillary services. We use the adversarial multi-armed bandit framework to learn the signal response of TCLs and determine which TCLs to activate for demand response in real-time. We demonstrate the performance of our proposed approach by invoking theoretical bounds on the performance of an Exp3.M-based algorithm, and comparing the performance with a greedy algorithm. A sub-linear regret shows that the algorithm is able to learn and identify high-performing TCLs, and activate them more frequently as more information is acquired about the TCLs' signal response.