A Lightweight Dual Stage Distillation Method for Knowledge Graph Completion

In the field of knowledge graph completion, traditional methods mainly rely on Knowledge Graph Embedding (KGE), which maps entities and relations into a low-dimensional vector space to predict missing links in the knowledge graph. However, as the embedding dimensions increase, the performance improvement is limited, while the computational resources and storage requirements significantly grow, making lightweight re-search for low-resource challenging. To address this issue, this paper proposes an innovative dual stage distillation that combines traditional knowledge distillation with self-distillation, dividing the iterative training of the model into a continuous sequence of iterations, with each pair forming a dual stage distillation unit, allowing the model to accumulate and optimize knowledge across continuous training periods. In addition, we propose a dynamic sample weight allocation mechanism, adjusting the weights of samples based on their quality, ensuring that high-quality sam-ples receive more attention and reducing the negative impact of low-quality samples. A series of experiments demonstrate that our proposed method outperforms existing baseline models on most metrics, especially in terms of accuracy metric (MRR), where we achieved state-of-the-art results.