On-line learning using hierarchical mixtures of experts

In the hierarchical mixtures of experts (HME) framework, outputs from several function approximators specializing in different parts of the input space are combined. Fast learning algorithms derived from the expectation-maximization algorithm have previously been proposed, but they are predominantly for batch learning. In this paper, several online learning algorithms are developed for the HME. Their performance in a piecewise linear regression task are compared according to criteria such as speed of convergence, quality of solutions, and storage and computational costs.