Combinatorial fusion with on-line learning algorithms

We give a range of techniques to effectively apply on-line learning algorithms, such as Perceptron and Winnow, to both on-line and batch fusion problems. Our first technique is a new way to combine the predictions of multiple hypotheses. These hypotheses are selected from the many hypotheses that are generated in the course of on-line learning. Our second technique is to save old instances and use them for extra updates on the current hypothesis. These extra updates can decrease the number of mistakes made on new instances. Both techniques keep the algorithms efficient and allow the algorithms to learn in the presence of large amounts of noise.