Sequential prediction of binary sequence with side information only

A simple on-line procedure is considered for the prediction of a binary-valued sequence in the setup introduced and studied by Weissman and Merhav [2001], [2004], where only side information is available for the algorithm. The (non-randomized) algorithm is based on a convex combination of several simple predictors. If the side information is also binary-valued (i.e. original sequence is corrupted by a binary sequence) and both processes are realizations of stationary and ergodic random processes then the average of the loss converges, almost surely, to that of the optimum, given by the Bayes predictor. An analog result is offered for the classification of binary processes.