On Limited Memory Universal Classification of Individual Sequences

We discuss a device called "classifier" (or discriminator) that observes an individual "training" sequence of length of m letters, X
^m

₁
. The classifier's task is to examine individual test sequences of length N and decide whether the test N-sequence has the same features as those that are captured by the training sequence, or is sufficiently different, according to some appropriate criterion. No a-priori information about the test sequences is available to the classifier, aside from the training sequence. A universal classifier d(N,Z
^N

₁
isin A
^N
) for N-vectors is a mapping from A
^N
onto {0,1}. Upon observing Z
^N

₁
, the classifier declares Z
^N

₁
to be "similar" to the training sequence X
^m

₁
, iff d(N,Z
^N

₁
) = 1. A trivial way to proceed is to compare the test sequence X
^N

₁
with each of the distinct N-vectors that appear in the training sequence X
^m

₁
. However, this procedure calls for a storage-space complexity that may grow exponentially with N. It is demonstrated that a particular universal context-tree classifier with a computational and storage complexity that is linear in N is essentially optimal. Another task for an N-sequence classifier is to decide whether two N-sequences Y
^N

₁
and Z
^N

₁
, or more, are similar to each other in the sense that each of the N-sequences is similar (as defined above) to the same unknown training sequence which is not available, (e.g "do these N-sequences sequences have a "common ancestor" - a frequent topic in computational biology). It is demonstrated that such an hypothesis may be essentially optimally tested based on the same context-tree classifier. This contributes a theoretical "individual sequence" justification for the probabilistic suffix tree approach in computational biology