KD-CB-MIA: A CNN-BiLSTM Collaborative Model for Membership Inference Attacks with Knowledge Distillation

Membership inference attack (MIA) has become one of the significant threats in the field of machine learning privacy and security. It works by determining whether a specific sample belongs to the training set of a target model. Existing black-box MIA methods face two major challenges: 1) The differences in output probability distributions between shadow models and target models lead to insufficient attack efficiency. 2) Traditional attack models have limited capability in extracting features from the output patterns of deep neural networks. In this work, we propose an enhanced attack framework that integrates knowledge distillation with a CNN-BiLSTM architecture, namely KD-CBMIA. Firstly, the training process of the shadow model is optimized through knowledge distillation, using the target model as a teacher model to guide the shadow model (student model) in learning the output distribution characteristics of the target model. Secondly, a hybrid attack model combining convolutional neural networks (CNN) and bidirectional long short-term memory (BiLSTM) is constructed. The CNN efficiently captures the confidence differences among different classes in the target model's output, while the BiLSTM models the correlations between dimensions in the logits vector of the target model's output. KD-CB-MIA addresses the insufficient representation of complex logits patterns by traditional attack models, improving attack accuracy. Experimental results demonstrate that the proposed attack model framework performs exceptionally well in membership inference attacks.