Universal schemes for learning the best nonlinear predictor given the infinite past and side information

Let {X/sub t/} be a real-valued time series. The best nonlinear predictor of X/sub 0/ given the infinite past X/sub -/spl infin///sup -1/ in the least squares sense, is equal to the conditional mean E{X/sub 0/|X/sub -/spl infin///sup -1/}. Previously, it has been shown that certain predictors based on growing segments of past observations converge to the best predictor given the infinite past whenever {X/sub t/} is a stationary process with values in a bounded interval. The present paper deals with universal prediction schemes for stationary processes with finite mean. We also discuss universal schemes for learning the conditional mean E{X/sub 0/|X/sub -/spl infin///sup -1/Y/sub -/spl infin///sup -1/Y/sub 0/} from past observations of a stationary pair process {(X/sub t/, Y/sub t/)}, and schemes for learning the repression function m(y)=E{X|Y=y} from independent samples of (X, Y).