EMMU: Efficient Information-Level Multimodal Machine Unlearning with High Model Fidelity

To comply with the “right to be forgotten,” recent research has introduced machine unlearning techniques that enable machine learning models to remove specific data samples. However, existing multimodal machine unlearning has concerns about efficiency, and the model fidelity may deteriorate after unlearning, leading to meaningless outputs if given data samples that are requested to be forgotten. Instead of focusing on datalevel machine unlearning, we focus on information-level unlearning, aiming to forget specific information of data samples (such as sensitive information involving name or medical condition) while maintaining the model fidelity. In this work, we design an efficient multimodal machine unlearning (EMMU) framework to address model fidelity in vision-language systems. The core idea is to locate and only modify model parameters that are highly correlated with the specific information (which requires forgetting). EMMU locates crucial parameters associated with the sensitive information and updates these parameters using a multiobjective optimization strategy. Evaluations on vision-language tasks, such as Visual Question Answering and Image Captioning, using multiple datasets, demonstrate the efficiency and fidelity of EMMU. Compared to existing methods, our method obtains an average of $111 \times$ improvement, boosting efficiency up to $1445 \times$. Meanwhile, the average utility-forget balance score has improved $9 \times$ on average and reached up to $70 \times$, across multiple models and datasets.