Change Point Detection Approach for Online Control of Unknown Time Varying Dynamical Systems

We propose a novel change point detection approach for online learning control with full information feedback (state, disturbance, and cost feedback) for unknown time-varying dynamical systems. We show that our algorithm can achieve a sub-linear regret with respect to the class of Disturbance Action Control (DAC) policies, which are a widely studied class of policies for online control of dynamical systems, for any sub-linear number of changes and very general class of systems: (i) matched disturbance system with general convex cost functions, (ii) general system with linear cost functions. Specifically, a (dynamic) regret of
$\Gamma_{T}^{1/5}T^{4/5}$
can be achieved for these class of systems, where
$\Gamma_{T}$
is the number of changes of the underlying system and
$T$
is the duration of the control episode. That is, the change point detection approach achieves a sub-linear regret for any sub-linear number of changes, which other previous algorithms such as in [1] cannot. Numerically, we demonstrate that the change point detection approach is superior to [1] and to standard online learning approaches for time-invariant dynamical systems. Our work presents the first regret guarantee for unknown time-varying dynamical systems in terms of a stronger notion of variability like the number of changes in the underlying system. The extension of the present work to state and output feedback controllers is a subject of future work.