Online Continual Learning Using Enhanced Random Vector Functional Link Networks

We propose an online continual learning algorithm based on an enhanced Random Vector Functional Link Network (OCL-eRVFL), that learns a sequence of tasks continually, where each task is defined by streaming data with each sample arriving once and only once. As data for a new task in domain incremental or class incremental setting streams in, the output weights of an eRVFL is updated through Recursive least squares, such that the representations for the past tasks are not catastrophically forgotten. As the recursive least square update is based only on the currently streaming sample, samples are not stored. Hence, unlike state-of-the-art OCL that avoid catastrophic forgetting through memory replay of samples from past task, the proposed OCL-eRVFL needs no extra memory. The proposed OCL-eRVFL is evaluated on streaming split CIFAR10, split CIFAR100 and split CIFAR10/100 image classification data sets within a class and domain incremental setting. Performance results show that the proposed OCL-eRVFL efficiently learns a sequence of tasks with streaming data, without additional memory expense.