SPARK: Simple and Parameter-Free Knowledge Embedding With Fuzzy Cognitive Maps for Class Incremental Learning

Class incremental learning (CIL) aims to mitigate catastrophic forgetting of previously learned classes when integrating new knowledge. A primary challenge contributing to forgetting is the absence of data from earlier classes. Researchers have designed a variety of methods to solve the problem, among which topology-preserving methods show tremendous potential. However, two problems remain: first, a large hyperparameter search space for constructing and utilizing a complex topology hinders efficient performance optimization, and second, constraining the network to preserve the topology in the objective makes it difficult to optimize. This article proposes SPARK, a simple and parameter-free method by embedding fuzzy cognitive maps, to address the problems. First, we construct a fuzzy cognitive map with nodes representing class prototypes and edges representing interclass similarities. Then, we exploit the fuzzy cognitive map to obtain class-level embedding by aggregating features of other classes for each class. Finally, the class- and sample-level embeddings are fused and fed to the classifier. The proposed method can be easily optimized without introducing additional loss terms and hyperparameters. We theoretically prove that such a simple fuzzy cognitive map embedding can efficiently preserve the structural information of the fuzzy cognitive map. Experimental results indicate that SPARK achieves up to 5.85% higher average accuracy and 8.69% reduction in forgetting compared to baseline methods.