Class Incremental Learning with Forward Memory

Prevalent dynamic network methods must provide additional task-IDs during inference with respect to class incremental learning. Otherwise, the classification accuracy will be significantly reduced. To tackle this issue, this work proposes a novel Class Incremental Learning with Forward Memory (CIL-FM) algorithm based on multi-head networks. The method learns and stores previously learned knowledge by introducing a forward branch, which then applies to new branches learning. A series of experiments confirm that forward branching can significantly improve the performance of dynamic network structures without task-IDs during inference. CIL-FM performs similarly to static network structures but excels in its continuous expansion as dynamic network structures without relying on task labeling cues in the inference phase. CIL-FM has better learning potential than static network structures.