Structure-constrain Source Free Domain Adaptation with Regularized Knowledge Distillation of Source Model

Transfer learning aims to leverage the knowledge of the source domain to help the learning of unlabeled target domain models. However, all conventional transfer learning methods assume that samples from source and target domains are always freely accessible. However, this assumption is rarely plausible in real-world due to data privacy. To combat this challenge, we introduce Structure-constrain Source Free Domain Adaptation with regularized Knowledge Distillation of source model (SSFDA-KD). Our method is based on the observations that a deterministic classification result can be obtained when the target data distribution aligns with the source data distribution, and a well-trained model not only yields a deterministic classification for a sample but also has clear category boundaries, which implies samples that are close together belong to the same category. From those, we first generate the source model with labeled source data, which is regarded as a well-trained model. Then, we identify target prototypes with lower self-entropy and assign the pseudo-labels to other samples according to the neighborhood structure. Finally, we refine the target model by minimizing prediction loss, constraining internal structure via triplet loss, and reducing knowledge distillation loss. Experiments of fault diagnosis based on WWTPs are conducted to validate its effectiveness.