Detecting Model Changes and their Early Warning Signals Using MDL Change Statistics

This study is concerned with the issue of detecting changes in Gaussian mixture models (GMM) from stream data. A change in a GMM implies a change in the number of clusters as well as a change in the cluster assignments. Unlike all the existing work that addresses this issue, we aim at detecting the early warning signals of changes as well as the changes themselves. To this end, we propose a novel notion of the sequential MDL change statistics (SMCS). SMCS is a real-valued index measuring the degree of change in a GMM from an information-theoretic viewpoint. This index can also be calculated sequentially every time a dataset is generated. Therefore, by tracking the changes of SMCS in real-time, we can possibly detect the early warning signals of changes in a GMM as well as its changes. We derive error probabilities for model change detection with SMCS to show that they converge to zero exponentially as sample size increases, and we derive a suitable parameter using this theorem. Furthermore, we empirically demonstrate cases where early warning signals of the changes can be successfully tracked by SMCS.