Identification of switching dynamical systems using multiple models

A switching dynamical system is a composite system comprising of a number of subsystems, where, at every time step, there is a certain probability that a particular subsystem will be switched on. Identification of the composite system involves: (a) specifying the number of active subsystems, (b) separating the observed data into groups, one group corresponding to each subsystem, (c) training a model for each subsystem and (d) combining the subsystems to form a model of the switching system. We use the term data allocation to describe steps (a) and (b); in case accurate data allocation is available (for instance using prior information, labeled data etc.), then efficient methods are available for performing steps (c) and (d). In this paper, however, we discuss the case where data allocation is not available and steps (a) and (b) must be performed concurrently with (c) and (d). This is, essentially, a problem of unsupervised learning. We present here conditions sufficient to ensure the convergence of a quite general class of data allocation schemes and relate these conditions to PAC learnability. The theoretical conclusions are supported by numerical experiments on a problem of online switching system identification.