Impacts of Catastrophic Forgetting: From a Machine Learning Engineering Perspective

Software validation is the method of ensuring that a software system is built correctly, meets user needs, and functions according to defined requirements. In the current age, Machine Learning (ML) and Deep Learning (DL) are at the forefront of technological innovation. Software engineering practices are being adopted in the ML/DL domain to assist in defining and mapping software validation anomalies, e.g., catastrophic forgetting. A ML model can be engineered, trained, and used to fulfill a requirement specification based on its classification capabilities. Catastrophic forgetting can cause a model to forget a class fully or partially insofar as a requirement is no longer fulfilled. This paper posits that catastrophic forgetting is an issue of ML validation and presents the issues of catastrophic forgetting in the novel domain of ML engineering. Ideally, this paper will spark a discussion and lend insight to the future direction of ML validation and verification. Future work includes developing case studies to observe the validation and verification challenges of catastrophic forgetting. The authors suggest researchers explore other ML/DL anomalies that might impact validation and verification.