Sequential nonlinear regression via context trees

In this paper, we consider the problem of sequential nonlinear regression and introduce an efficient learning algorithm using context trees. Specifically, the regressor space is partitioned and the resulting regions are represented by a context tree. In each region, we assign an independent regression algorithm and the outputs of the all possible nonlinear models defined on the context tree are adaptively combined with a computational complexity linear in the number of nodes. The upper bounds on the performance of the algorithm are also investigated without making any statistical assumptions on the data. A numerical example is provided to illustrate the theoretical results.