Inter-class and Intra-class Relationships Incorporated Knowledge Distillation for Continual Learning

Continual learning enables models to learn sequentially from a stream of tasks while retaining previously acquired knowledge. However, current methods lack the attention of tasks with different categories, as new task is introduced, models often adjust their internal representations to accommodate the new knowledge, which leads to decision boundary shifts and catastrophic forgetting, limiting the performance of continual learning methods. To address these limitations, this paper proposes continual learning with inter-class and intraclass relationships incorporated knowledge distillation (2ICL). The inter-class ensures stable decision boundaries by capturing the relative positioning between task categories. Intra-class relationships preserve internal coherence within each class to enhance generalization. Furthermore, 2ICL incorporates a dynamically expandable representation, enabling it to expand its feature space as new tasks are added while retaining old and new knowledge. Experiments conducted on CIFAR-10, CIFAR-100, and Pathmnist datasets demonstrate that 2ICL not only significantly alleviates catastrophic forgetting but also maintains high accuracy across tasks.