Reinforcement Learning for Sustainability: Adapting in large-scale heterogeneous dynamic environments

Reinforcement learning (RL) has seen major breakthroughs in the recent years, most notably outperforming humans Atari, Go, and StarCraft games. RL use is also being investigated in a range of practical applications, from medical diagnostic to news personalization, and in particular in self-adaptive systems [1]. In this talk, we focus on applications of RL for optimization of resource use in city infrastructures [2], for example, urban traffic control [3], smart grid [4], and communication networks [5], [6], aimed at improving their sustainability and addressing the ever-increasing demand on urban resources. We first analyze properties that make these systems ameanable to RL-based optimization, and then present a range of techniques we developed for enabling the use of RL for optimization in such environments, namely, multi-agent multi-objective optimization, adaptation in non-stationary environments, online transfer learning, state space adaptation, and RL explainability. We close by discussing further challenges in enabling RL deployment in self-adaptive large scale infrastructures, including development of new algorithms that can ensure seamless lifelong adaptivity, including adapting to previously unseen situations.