Timespan-Aware Dynamic Knowledge Graph Embedding by Incorporating Temporal Evolution

Recently, Knowledge Graph Embedding (KGE) has attracted considerable research efforts, since it simplifies the manipulation while preserving the inherent structure of the KG. However to some extent, most existing KGE approaches ignore the historical changes of structural information involved in dynamic knowledge graphs (DKGs). To deal with this problem, this paper presents a Timespan-aware Dynamic knowledge Graph Embedding Evolution (TDG2E) method that considers temporal evolving process of DKGs. The major innovations of our paper are two-fold. Firstly, a Gated Recurrent Units (GRU) based model is utilized in TDG2E to deal with the dependency among sub-KGs that is inevitably involved in the learning process of the dynamic knowledge graph embedding. Furthermore, we incorporate an auxiliary loss to supervise the learning process of the next sub-KG by utilizing previous structural information (i.e., the hidden state of GRU). In contrast with existing approaches in the literature (e.g., HyTE and t-TransE), TDG2E preserves structural information of current sub-KG and the temporal evolving process of the DKG simultaneously. Secondly, to further deal with the time unbalance issue underlying the DKGs, a Timespan Gate is designed in GRU. It makes TDG2E possible to model the temporal evolving process of DKGs more effectively by incorporating the timespan between adjacent sub-KGs. Extensive experiments on two large temporal datasets (i.e., YAGO11k and Wikidata12k) extracted from real-world KGs validate that the proposed TDG2E significantly outperforms traditional KGE methods in terms of Mean Rank and Hit Rate.