Memory Enhanced Replay for Continual Learning

Currently, artificial intelligence can not continually learn the new knowledge and retain the acquired knowledge in the past like human beings. It is affected by a phenomenon called catastrophic forgetting. Although continual learning based on generative replay could alleviate the problem to a certain extent without accessing the past data, it utilizes not fully potentials of the framework. Inspired by the mechanism of the interaction between the brain hippocampus and neocortex, we propose a novel memory enhanced replay networks (MER) based on generative replay for continual learning. This is a generative adversarial networks (GANs) integrating memory enhancement module. Specifically, the method adopts GANs to simulate the learning and memory process in the brain and add a memory enhancement module in the final process of training. This approach allows the network to relieve the uneven distribution of tasks in the previous sequence training, so as to better preserve the past learned knowledge. Class-incremental experiments show that the proposed framework is able to promote knowledge consolidation after training and has a competitive result compared with other advanced methods.