Research on Neural Graph Collaborative Filtering Recommendation Model Fused With Item Temporal Sequence Relationships

Graph neural network-based recommender systems are blossoming recently, and it can explicitly express user-item high-order connectivity information, so it can significantly improve the recommendation performance. However, the existing methods usually assume that the users’ interests are invariant, but temporal relationships is left insufficient exploration to charactertize the user’s dynamic interest. In this paper, we propose a Neural Graph Collaborative Filtering recommendation model fused with Item Temporal Sequence relationships (NGCF-ITS), that is, a hybrid recommendation model that fuses with user-item interactions information and item temporal sequence relationships. It divides the item temporal sequences into several groups of subsequences through the sliding window strategy, then constructs the item temporal sequence relationships graph and aggregates the characteristics of item temporal sequences information. At the last, deeply depicts the dynamic changes of users’ interests, and uses the bipartite graph neural network to map the high-dimensional information of user-item and item-item into the low-dimensional space. The hybrid embedding of user-item historical interactions and item temporal sequence relationships are realized, and the expression of user-item interactions is enhanced. In this way, the heterogeneous multi-relational graphs are fused for the feature propagation, which largely refines the user and item representation for model prediction. Extensive experiments demonstrate the our proposed model significantly improve the recommendation performance compared to the state-of-the-art GNN-based models both in accuracy and training efficiency.


I. INTRODUCTION
Recommender systems have been widely applied to daily life such as online shopping, social networking, advertising recommendation and Web search. It can effectively avoid information overload and provide users with accurate personalized information recommendation services. For each user, the goal of the recommender systems is to predict what goods he or she may buy in the future [1], [2], [3]. Collaborative Filtering The associate editor coordinating the review of this manuscript and approving it for publication was Wei Jiang .
(CF) [4], [5], [6] is a classic recommend method to dig users' interests by users' historical behaviors. The core idea of the traditional CF is to recommend users' favorite items with the same preferences [7], [8], such as User-Based CF [9] and Item-Based CF [10], [11]. However, efficient CF-based methods largely depend on the richness degree of the useritem interactions. Therefore, they have poor performance in some scenarios where item information or interactive records are sparse and insufficient, which often leads to the cold start problems in practical marketing [12]. To address the above problems, some researchers have proposed some hybrid recommendation models [13], [14], [15] to fuse with multi-source heterogeneous auxiliary information, which can alleviate the problems of cold start and data sparsity. However, the auxiliary information is one of the main challenges building such hybrid recommender methods, such as multimodal, heterogeneous, large-scale, sparse and uneven distribution. On these basis, most CF-based recommender methods mapped user-item interations into two lowdimensional spaces, for example, Matrix Factorization (MF) [16] and Neural Network [17]. To obtain high-quality embedding representations, most researchs apply graph neural networks (GNNs) to embed user-item interactions, which significantly improves the overall recommendation performance. Tang et al. proposed the unsupervised network embedding algorithm Line [18] and node2vec [19]. Ying et al. [20] proposed a recommendation model PinSage based on GNNs, by adopting methods such as random walk and graph convolution to capture the features of graph structure and nodes to generate embedded representations of nodes, which greatly improves the quality of embedded representation. Wang et al. [21] developed a graph neural collaborative filtering recommender method NGCF, which embedded and encoded the user-item historical interaction information into a bipartite graph, and explicitly consided the high-order connectivity between users-items to further improve the embedded presentation ability. He et al. [22] proposed a light weight graph convolutional network model, LightGCN, for recommender systems, which abandoned the feature transformation and nonlinear activation of traditional GNNs, through propagation to learn user and item embeddings, and finally the weighted summation of the embeddings learned by all layers is used as the final embedding representation. Zhang et al. [23] argued a hybrid recommendation method based on the combination of paragraph embedding and neural network. This method uses neural network to collaborative filter the user's item score, and has high nonlinearity to capture the complex structure of user interaction score. Moreover, the problem of cold start is relieved by using the content characteristics of item embedding to obtain auxiliary information. Wang et al. proposed KGCN [24], which combined knowledge graph and recommendation with GNNs to dig the relationship of goods on knowledge graph, and effectively capture the internal relationship of goods and alleviate data sparsity. Sang et al. [25] proposed a knowledge graph-enhanced neural collaborative filtering model KGNCF-RRN, which utilized knowledge graphs and long-term relationships of user-item interactions for recommendation, which effectively solved the sparsity problem of recommender systems. Tang et al. [26] proposed a multi-graph collaborative filtering model (DMGCF) based on a dynamic evolution mechanism, constructed user graph and item graph through user-item bipartite graph and embedding, simulated the edge information in the latent space, and proposed a new dynamic evolution mechanism for collaborative updating during learning and improved embedding and graph structure for better embedded content.
However, most researchs on GNNs only focus on the user-item binary relationships, without considering internal logic relationships between the user-item interactions and the item temporal sequence relationships. The existing methods usually assume that the users' interests are invariant, but temporal relationships is left insufficient exploration to charactertize the user's dynamic interest. While intuitively useful to integrate user-item interactions into the embedding function, it is non-trivial to do it well. In particular, the scale of interactions can easily reach millions or even larger in real applications, making it difficult to distill the desired collaborative signal. Also, most methods use the descriptive characteristics of users or items to build embedded functions. When the relationships between users and items becomes greatly complex, these methods will lead to over-fitting, data sparsity and lower recommendation performance. On the other hand, the existing researches focused on users' shortterm interests, ignored the impact of item temporal sequences on users' long-term interests or implicit interests. To solve these problems, this paper proposes a hybrid recommendation model, NGCF-ITS, which integrates user-item interactions and item temporal sequence relationships. The illustration of our recommendation model is shown in Figure 1. Our contributions can be summarized as the following: 1) We propose a top-N hybrid recommendation model that integrates user-item interaction information and item temporal sequence relationships. We adopt the sliding window strategy to divide the item temporal sequences into several groups of subsequences, build the item temporal sequence relationships graph, and aggregate the information features of the item sequence to describe deeply the dynamic interests.
2) Apply the bipartite graph neural network to embed the user-item historical interactions and item temporal sequence relationships, and map the user-item and item-item highorder information into the low-dimensional space to enhance the user-item interactions and improve the recommendation performance.
3) Construct a neural graph collaborative filtering recommendation model, the model mainly includes four layers. The four layers refer to the embeddding layer, aggregation layer, propagation layer and prediction layer. 4) We conduct empirical studies on three standard millionsize datasets, LastFM, Ciao and Douban. Extensive results demonstrate the state-of-the-art performance of NGCF-ITS.

II. PRELIMINARIES
This section introduces the problem definition and flowchar of the NGCF-ITS. To facilitate discussion, some symbols are introduced with specific definition as shown in Table 1. The flowchart of the NGCF-ITS is illustrated in Figure 2.
A. PROBLEM DEFINITION Some symbols are introduced with specific definition as shown in Table 1. Similar to other recommendation algorithms, let P = [p 1 , p 2 , p 3 , . . . , p M ] and Q = [q 1 , q 2 , q 3 , . . . , q N ] be user embedding matrix and item VOLUME 10, 2022  embedding matrix respectively, with p u ∈ R d , (q i ∈ R d ) is embedding vector of user u (item i), d is the dimension in NGCF-ITS, P ∈ R d×M , Q ∈ R d×N .
In addition, most existing neural graph network recommendation algorithms describe the user-item interactions by establishing the bipartite graph, and capture the embedding of users and items by utilizing the graph structure methods. The traditional aggregation methods play a crucial role in the message transmission mechanism. However, previous studies only aggregated adjacent information from the perspective of spatial structure information. These methods usually assumed that users' interests are static, and less considered users' dynamic interests changing through the item temporal sequences. Therefore, we construct item-item temporal sequence relationships graph to capture users' dynamic interests changing. Here, we define the item-item directed graph G < V , E >, where V is item set, E shows edge set of items. < V x , V y > represents an edge between item x and item y. Finally, we create user-item interaction graph G < u, r, i >, where u represents user set, i denotes item set, r denotes user-item interaction sitution, Where r = 1 means this interaction is considered to be a success, or otherwise that is a failure.

B. NGCF-ITS
We now present the proposed NGCF-ITS model, the flowchart of which is illustrated in Figure 2. There are four layers in the flowchart. The detailed working mechanism of each layer is presented as follows.
1) Embedding layer: An embedding layer that offers and initialization of user embeddings and item embeddings. After being initialized in the embedding layer, the user-item potential collaborative signals will be capured in the subsequential training process of the recommendation model Moreover, the node feature in graph will be mapped into low-dimensional space, the embeddings will be iteratively optimized. (step 1) 2) Aggregation layer: To describe users' dynamically interests changing, we design a sliding window strategy which to split the item sequence into fine-grained sub-sequences.
Then, building the item temporal sequence relationships graph, we aggregate the characteristics of item temporal sequences in the graph. Aggregation layers that refine the embeddings by injecting high-order connectivity relations, the aggregation method can find latent features for both users and items similarly. (step2, 3) 3) Propagation layer: We devise an embedding propagation layer, which refines a user's (or an item's) embedding by aggregating the embeddings of the interacted items (or users). By stacking user-item interaction graph and the item-item temporal sequence relationships graph, we can enforce the embeddings to capture the collaborative signal in high-order connectivities. (step 4, 5) 4) Prediction layer: The prediction layer that aggregates the refined embeddings from different propagation layers and outputs the affinity score of a user-item pair, and output prediction results of recommendation. (step 6)

III. METHODS
In this section, We will describe our proposed model research on neural graph collaborative filtering recommendation model fused with item temporal sequence relationships (NGCG-ITS). As shown in figure 2, there are four layers: Embedding layer, Aggregation layer, Propagation layer, Prediction layer.

A. EMBEDDING LAYER
In particular, the scale of user-item interactions can easily reach millions or even larger in real application, it is difficult to distill the desired collaborative signal. In this work, we tackle the change by employing embedding methods based on graph neural network, P and X can be initiated randomly, the user embedding matrix P = [p 1 , p 2 , p 3 , . . . , p M ], where M is the number of users and the item embedding matrix Q = [q 1 , q 2 , q 3 , . . . , q N ] with N is the number of items, P ∈ R d×M , Q ∈ R d×N , where d is embedding dimension, we describe a user u (an item i) with an embedding vector p u ∈ R d (q i ∈ R d ).
We apply the bipartite graph neural network to embed the user-item historical interactions and item temporal sequence relationships, and map the user-item and item-item highorder information into the low-dimensional space to enhance the user-item interactions and improve the recommendation performance.

B. AGGREGATION LAYER
Prior works of GNN-based recommendation only focus on static information of user's interest but ignore the item temporal sequence information. Thus, We describe users' dynamic interests by aggregated embed user-item history interactions and the item temporal sequence relationships to relieve gradient extinction. On the one hand, users' short-term interests and long-term interests shouled be considered, which can help better reflect a user's interest and alleviate the sparsity issue of user-item interactions. On the other hand, to relieve the computing pressure from users' historical interations and item temporal sequence relationships, we adopt sliding window strategy to construct temporal sequence relationships graph. Firstly, we conduct a sliding window strategy to split the item sequence into fine-grained sub-sequences. Then, design a item-item temporal sequence relationships graph for learning the sub-sequences of the same item. At the last, we describe users' interest changing by aggregating historical interactions and temporal sequence relationships. The main description is as follows: We incorporate the item-item sequence relationships graph to better indicate the user's interests and alleviate the user-item sparsity issue. Hence, we define the item-item directed graph . . , V M } denotes the sequence set of items interactived by users. By splitting V into sub-sequences T = {T 1 , T 2 , T 3 , . . . , T H }. Then, we construct item-item temporal sequence relationships graph for learning the sub-sequences of the same items. To overcome the sampling simply, which will lose some important features of user's interest and diversity of item, we use β to denote the length of a subsequence. In the section, to demonstrate the effectiveness of β, we conduct comparative examples with different kinds of parameters setting, including β = 2 and β = 3, the detailed process is shown in Figure 3. For example, in Figure 3(a), while β = 3, the sequence set of user-item historic interactions is Next, Figure 3(b) illustrates the connection process of item-item temporal sequence relationships graph. where N 1 means the number of connected edges, and N 2 represents the number of unconnected edges. As shown in the Figure 3, to analyze the benefits of sliding window method, we vary the number of β in the range of 2, 3, the obvious observation is that the performance tends to better during the increase of β. For example, β = 2, N 1 = 4, N 2 = 6, and β = 3, N 1 = 7, N 2 = 3. The detailed analysis process will be discussed in the experiment section.

2) AGGREGATE ITEM-ITEM TEMPORAL SEQUENCE RELATIONSHIPS
With the item temporal sequence relationships explosion, the proposed model has become important for embedding user-item interactions. Graph Convolutional Network (GCN) can not only process topological graph structure data, but also iteratively aggregate feature information from neighbor nodes and excavate deeper potential features and internal laws. Therefore, we utilize GCN to aggregate item temporal sequence relationships and capture the higher-order relationships of items by multiple aggregation embedding. The l-th item embedding vector of aggregation layer is: whereq l v is item v embedding vector, when In this way, we implement m v− →v as: LeakyReLU () represents activate function. The normalized matrixĀ = ϕ(A + E), where A and E are item connection matrix and unit matrix, respectively. ϕ() is normalized functions.Ā v,v denotes the normalized node weight value of item v regarding item v . Item embedding vector of aggregation layer is: W l 0 and d l 0 represent the trainable weight matrices. We consider not only the temporal sequence relationships of items from neighbor node, but also the characteristics of items.

C. PROPAGATION LAYER
In recommender systems, the user-item historical interactions are dominant in model prediction. Hence, we define the user-item interaction graph as G < u, r, i >, where u and i denote the user and item set respectively, r denotes the situation of user-item inter-action, Where r = 1 means success interaction, otherwise that is a failure. Therefore,We apply user-item connection to capture cooperative signals and obtain high order similarity. The user embedding vector of l-th is: where p l u represents embedding vector of user u during l-th propagation. Note that in addition to the messages propagated from neighbors U v , we take the self-connection of u into consideration: m l u← −u = W l u,1 p l−1 u , which retains the information of original features. m l u← −u and m l u← −v denote weight matrix. LeakyReLU () is activate function, which allows messages to encode both positive and negative signals. m l u← −v is: We define graph Laplacian norm as 1 √ |N u ||N v | , where N u , N v denote the first-hop neighbors of user u and item v, respectively. From the view point of representation learning, it reflects how much contributes of the historical item to the user preference. From the view point of message passing, it can be interpreted as a discount factor, considering the messages being propagated should decay with the path length. To increase the expressiveness of the model, we utilize embedding informationq l−1 v and use-item interactive informationq l−1 v ⊕ p l−1 u . In addition, W l u,1 , W l u,2 ∈ R d×d is the weigh matrix.
Similar to user embedding vector, the embedding vector of the item can be written as:

D. PREDICTION LAYER
When the aggregation layer and propagation layer are aggregated and propagated for many times, we will find that multiple operations can not only learn the dependence of item temporal sequence relationships, but also explore the higherorder connectivity. We observe how the learning results are influenced in experiment, since the representations obtained in different layers emphasize the messages passed over different connections, they have different contributions in reflecting user preference. So, we connect p 1 u , p 2 u , p 3 u , . . . , p l u , . . , q l v and get final vector of user embedding p u and vector of item embedding q v : where is concatenation operation, and we define a l-th vector of the user or item embedding. The range of l can not only effectively obtain the embedding vectors of users and items, but also avoid insufficient information extraction. A larger value indicates that the space and time complexity are higher, while a smaller value indicate the opposite. The final precision result is:ŷ We regard the observed user-item interactions as positive samples. otherwise, it is regarded as negative samples. The proposed model will be optimized by minimizing the Bayesian Personalized Ranking (BPR) loss. The BPR loss is based on the assumption that a user-item interaction that has already existed in the datasets should be assigned a higher ranking score than an interaction that does not exist in the datasets. The objective function is as follows.
where (u, v, z) in O, (v, z) represents the training sample sets, which indicates item v interactive with user u, (u, z) is the unobserved interaction. σ () is the sigmoid function, and 2 2 denotes the L 2 regularization term, which is to prevent over-fitting and represents model training parameters.

IV. EXPERIMENTS
We perform experiments on three real-world datasets to evaluate our proposed model, especially the aggregation layer.
We aim to answer the following research questions: 1) How does different embedding dimension d affect NGCF-ITS?
2) How do the layer number l of the aggregation layer and the sliding window parameter β affect the recommendation performance?
3) How does NGCF-ITS perform as compared with stateof-the-art models?

A. EXPERIMENTAL SETTINGS 1) DATASETS
To verify the unbiasedness of the model, we conduct experiments on three real-world datasets: LastFM, Douban, and Ciao. These datasets are all publicly accessible, independent  of each other, and vary in application domain, data size, and data sparsity.
The attributes of the three datasets are shown in Table 2, Ciao datasets obtiains higher data sparsity. The obvious observation is that the recommendation performance of the model under different sparsity levels. The details of the datasets are as follows: 1) LastFM: LastFM is an online radio and music community set in the UK. It provides developers with a wealth of APIs which including data of various sizes. We apply the datasets of user listening sequence with implicit feedback of context information released at the 2nd international seminar of the 5th ACM recommendation system conference.
2) Douban: Douban is a well-known theme social networking site in China. The topics include movies, books, music, etc. Users can conduct social discussions on topics of interest, and the content of the discussions can generate cross-information feedback for other users. We conduct experiments on a subset of movie ratings.
3) Ciao: Ciao is a DVD category datasets crawled from http://dvd.ciao.co.uk website in December 2013, which covering user ratings for various items and their personal reviews. The scoring mechanism is the scoring interval of [1], [5], and the average of the effective scores of the comprehensive user shows that the score and evaluation will affect the user's willingness to purchase the item.

2) EVALUATION PROTOCOLS
In this section, we take all-ranking protocol and adopt four evaluation metrics : Precision@N , Recall@N , NDCG@N . The detailed introduction is shown as follow:  In our experiments, the length of the recommendation N is in 5, 10, 20, 40.Ū represents all users of test set. R u denotes all item sets which were recommended by user u. I u means item sets which have interacted with user u. The precision, recall rate and NDCG under N recommended items, respectively denoted as Precision@N , Recall@N , NDCG@N .

3) PARAMETER SETTING
The experiment was conducted on tensorflow platform. In the initialization phase, the embedding dimension of the model is set to d = 64, and the batch processing is set to 1024. We initialize the parameters in the model with Xavier, set the learning rate and L 2 regularization to 0.0001 and 10 5 , respectively, and use the Adam optimizer for model optimization training. If the value of Recall@20 does not continue to increase for 50 consecutive epochs, the model stops training.

4) BASELINES
In order to test the performance improvement of the NGCF-ITS model, we conduct comparative experiments with some current advanced recommendation models. The models are introduced as follows: 1) APR [27]: Considering that when BPR [28] optimizes MF, the model parameters are susceptible to adversarial disturbances, APR combines BPR with adversarial training methods to make the recommendation model more robust, thus improving the accuracy of model recommendation.
2) CFGAN [29]: A novel CF model based on GAN, which can achieve higher recommendation system accuracy. By adjusting the score vector of the generated user of the G network in the adversarial network, the value range is 0 to 1, which solves the problem of discrete generation in the traditional IRGAN, and also optimizes for collaborative filtering.
3) CUNE_BPR [30]: The CUNE_BPR model extracts implicit social information from user feedback on items, and identifies top-N semantic meta-information for each user, incorporating top-N semantic metainformation into the BPR framework to solve the problem of project sequencing. 4) NeuMF [31]: The NeuMF model combines the linear features of MF and the nonlinear features of DNNS [32], [34], [35] to build latent structures between users and items, through multiple hidden layers on the element level and the embedded connections between users and items. 5) NGCF [21]: The NGCF model explicitly models the interaction information between users and items, constructs a user-item interaction graph, and on this basis, effectively embeds the collaboration signal into the user-item interaction graph propagation process to achieve high-order connectivity, which improves the embedded representation of users and items. 6) LightGCN [22]: LightGCN model simplifies NGCF model. It eliminates feature transformation and nonlinear activation in NGCF, because these designs have no positive impact on collaborative filtering, and reduces part of the noise of NGCF embedding. Compared with NGCF, LightGCN not only improves the operation efficiency, but also significantly improves the recommendation performance.

B. PERFORMANCE 1) IMPACT OF SLIDING WINDOW SIZE OF β
To investigate whether NGCF-ITS can benefit from sliding window size, we vary the β. In particular, we search the β in the range of 1, 3, 5, 7, 9. Figure 4 summarizes the experiment results, where indicates that the item-item sequence relationships and the users' dynamic interests are affected by sliding window β. We have the following observations in Figure 4: 1) Different sliding window size of NGCT-ITS influences the recommendation cases. NGCF-ITS generally achieves better performace in Douban (β = 5), Ciao (β = 5), and LastFM β = 7.
2) It can be observed that the optimal β is highly dependent on the average length of the neighbors of the datasets, so the optimal β can be set through the properties of the datasets.

2) IMPACT OF THE NUMBER OF LAYERS l
Parameter l controls the intensity of learning, and during the increase of l, the obvious observation is that the recommendation can caputre users' dynamic interests accurately.
To verify the performance of l on recommendation model, we set l in range of 0, 1, 2, 3, 4. Figure 5 shows that the experiment results on Recall@20, Precision@20, NDCG@20. NGCF-ITS obtains the worst performance on three datasets with l = 0, in other words, the item temporal sequence relationships is not considered. In contrast, the temporal sequence relationships can indeed improve recommendation performance.
Compared with LastFM and Douban, due to Ciao is very sparse, the influence of that is relatively modest, which will quickly speed up the convergence of the underlying model and offer better performance.
In addition, during the increase of l, the experiment results is relatively stable at the same time. For example, the result of LastFM and Douban is better with l = 3, Ciao is better with l = 2. Then, the experiment results declined, the reason might be that the value of l is higher, the feature learning of graph convolution networks on graph structure is too smooth. The learning of item temporal sequence relationships reaches the state of over-fitting, leading to a poor performance.

3) IMPACT OF EMBEDDING DIMENSION d
To analyze the importance of embedding dimension d on three datasets, The dimension d is a multiple of 8, such as d = [8,16,32,64]. The experiment data are shown in Table 3, the experiment performance is worst on three datasets with d = 8. During the increase of d, the experiment data on LastFM and Douban tend to better, the effect of rising is especially obvious in d = 8 to d = 16, then the changing is slowly in d = 64, the experiment results reach the best performence.
Different from LastFM and Douban, the experiment results reach the best in Ciao with d = 16. the reason for this phenomenon is that Ciao is relatively sparse. In this case, compared with high-latitude embedding, better results can be obtained by training low-dimension embedding, while high-latitude embedding will lead to over-fitting of data and lower recommendation performance, which is also explained in the paper [19].

4) COMPARION WITH REPRESENTATIVE MODELS
To evaluate our proposed model, the experiments are carried out on three real-world datasets, and to the accuracy performance of the propose model, we compare with other recommendation models, such as the improved CUNE_BPR and APR based on traditional BPRMF, NeuMF based on neural networks, CFGAN based on generative adversarial network, NGCF based on GNN and LightGCN model simplifies NGCF. The above model is described in BASELINES.
1) Table 4 shows the experimental results on LastFM, Douban and Ciao with N = 5, 10, 20, 40. we have the following observations: our proposed model NGCF-ITS typically outperformed approaches, which reflects the accuracy and effectiveness. In addition, based GCN model use graph structures for message propagation to achieve performance improvements.
2) NGCF-ITS aggregates item temporal sequence relationships, this not only increases the model representation ability, but also boosts the performance for recommendation, and NGCF-ITS generally achieves better performance than LightGCN in most cases.
3) NGCF-ITS consistently yields the best performance on all the datasets. In particular, NGCF-ITS improves over LightGCN is 4.0% on LastFM, over CUNE_BPR is 6.8% on Douban and over NeuMF is 5.2% on Ciao in Table 4. This phenomenon indicates that aggregation of item temporal sequence relationships into GNN-based recommendations has sufficient development prospects.

V. CONCLUSION
In this paper, we proposed a neural graph collaborative filtering recommendation model fused with item temporal sequence relationships (NGCF-ITS). Specifically, we utilized the sliding window strategy to divide the item temporal sequence relationships into several groups of sub-sequences, and constructed the item temporal sequence relationships graph to aggregate the item-item temporal sequence relationships and dig users' dynamic interests. We use bipartite graph neural network to map high-dimensional information of useritem and item-item into low-dimensional space to realize the hybrid embedding of user-item historical interaction information and item-item temporal sequence relationships information, expression of information can be enhanced by user-item interaction sequence. Moreover, make use of the user-item interaction graph and item temporal sequence relationships graph to construct a four-layer neural graph collaborative filtering recommendation model framework, including the embedding layer, the aggregation layer, the propagation layer and the prediction layer. The results of comparative experiments on the three real-world datasets of LastFM, Ciao and Douban, which shows that NGCF-ITS model has brought significant improvements compared with other models, meaning that the item temporal sequence relationships is valuable in providing more accurate recommendations.