Universal Compression of Piecewise i.i.d. Sources

We study the problem of compressing piecewise i.i.d. sources, which models the practical application of jointly compressing multiple disparate data files. We establish that universal compression of piecewise i.i.d data is possible by modeling the data as a Markov process whose memory grows suitably with the size of the data using the Krichevsky-Trofimov (KT) estimator. The memory order is chosen large enough so that successful learning of the distribution of the each piece of the data from the corresponding contexts is possible for almost any realization of any piecewise i.i.d. data process. This is, a priori, a surprising result given that we are employing a stationary model to asymptotically optimally (model and) compress non-stationary data.