Data-Driven Quickest Change Detection in Markov Models

The problem of quickest change detection in Markov models is studied. A sequence of samples are generated from a Markov model, and at some unknown time, the transition kernel of the Markov model changes. The goal is to detect the change as soon as possible subject to false alarm constraints. The data-driven setting is investigated, where neither the pre-nor the post-change Markov transition kernel is known. A kernel based data-driven algorithm is developed, which applies to general state space and is recursive and computationally efficient. Performance bounds on the average running length and worst-case average detection delay are derived. Numerical results are provided to validate the performance of the proposed algorithm.