Catastrophic Forgetting within AI-Generated Dataset

Catastrophic forgetting is an unavoidable phenomenon that arises when a previously learned model undergoes further training on a target dataset. When an event occurs, there is a loss of knowledge from past training, whether it be a partial or complete loss. In this study, there are two interconnected contributions. This research presents the introduction of a novel dataset for the purpose of examining the occurrence of catastrophic forgetting. The dataset was generated by employing a generative language model using an existing sentiment analysis dataset known as Sentiment140. The experiment yielded findings indicating that the created dataset exhibits distribution shifts in comparison to the original dataset. The present study involves the examination of two distinct language models, namely BERT and DistilBERT, which possess varying numbers of parameters. Moreover, the utilization of this information lends itself to the examination of domain adaptation. The occurrence of this phenomenon can be attributed to the domain shift that takes place between the two datasets.