Multiadvisor Reinforcement Learning for Multiagent Multiobjective Smart Home Energy Control

Effective automated smart home energy control is essential for smart grid approaches to demand response (DR). This is a multiobjective adaptive control problem because it balances an appliance’s primary objective with demand response objectives. One challenge comes from the heterogeneous nature of objectives, requiring tradeoffs between comfort, cost, and other objectives. Another challenge comes from the heterogeneous dynamics, which result from different environments and the different appliances used. Another challenge is nonstationary nature of dynamics and rewards due to seasonal changes and time-varying user preferences. Finally, we consider computational challenges, required by the real-time aspect of the control problem, particularly notable due to “the curse of dimensionality.” We propose a multiagent multiadvisor reinforcement learning framework to address these challenges. We design a smart-home simulation to demonstrate the performance (in terms of weighted reward) of our approach relative to competitive single-objective reinforcement learning algorithms. Furthermore, we theoretically and empirically demonstrate the linear computational scalability of the algorithm. Finally, we identify the need for key performance measures of the proposed system by considering the effect of selected preferences on agents. Overall, the proposed algorithm is reasonably competitive with conventional approaches while simultaneously enabling behavior changes with change in preferences without requiring more data.