Redundancy of the Context-Tree Weighting Method on Renewal and Markov Renewal Processes

The <emphasis emphasistype="italic">Context Tree Weighting</emphasis> method (CTW) is shown to be almost <emphasis emphasistype="italic">adaptive</emphasis> on the classes of <emphasis emphasistype="italic">renewal</emphasis> and <emphasis emphasistype="italic">Markov renewal</emphasis> processes. Up to logarithmic factor, <emphasis emphasistype="smcaps">ctw</emphasis> achieves the minimax pointwise redundancy described by I. Csiszár and P. Shields in <emphasis emphasistype="boldital">IEEE Trans. Inf. Theory</emphasis>, vol. 42, no. 6, pp. 2065–2072, Nov. 1996. This result not only complements previous results on the adaptivity of the Context-Tree Weighting method on the relatively small class of all finite context-tree sources (which encompasses the class of all finite order Markov sources), it shows that almost minimax redundancy can be achieved on massive classes of sources (classes that cannot be smoothly parameterized by subsets of finite-dimensional spaces). Moreover, it shows that (almost) adaptive compression can be achieved in a computationally efficient way on those massive classes. While previous adaptivity results for CTW could rely on the fact that any Markov source is a finite-context-tree source, this is no longer the case for renewal sources. In order to prove almost adaptivity of CTW over renewal sources, it is necessary to establish that CTW carefully balances estimation error and approximation error.