Structural Embedding Pre-Training for Deep Temporal Graph Learning

Thanks to the fact that many real-world data can be modeled as graph-structured data, graph deep learning is gradually attracting close attention from researchers. As an integral component of graph learning, temporal graph learning abandons the data format of adjacency matrices and instead utilizes interaction sequences to record and observe real-time changes in nodes. However, temporal graph learning both benefits from and is constrained by this approach. The data format of interaction sequences often leads temporal graph learning methods to focus only on the closest neighborhoods, making it difficult to capture high-order structural information and resulting in noticeable information loss. To ensure that temporal graph learning methods can consider both high-order structural information and maintain their flexibility, we propose SET, a temporal graph method which introduces Structural Embedding pre-training to enhance Temporal graph learning. Specifically, we achieve this by introducing classical methods to pre-train the data, generating node initialization embeddings that focus on high-order structural information. Furthermore, we constrain the model to optimize not only based on these embeddings but also to approach them as signals for data augmentation. By incorporating structurally embedded features through pre-training, we are able to obtain a broader receptive field without compromising model efficiency. We conducted experiments on several datasets, the experimental results validate the performance of our proposed method SET.