ReGNN: A Repeat Aware Graph Neural Network for Session-Based Recommendations

Session-based recommendations have attracted significant attention because of their broad application scenarios. Recently, graph neural networks (GNNs) have been employed in session-based recommendation because of their superior performance in representation learning compared with recurrent neural networks (RNNs). Although most existing GNN-based methods have made great achievements in this field, none of them emphasizes the importance of repeat recommendations, which has been an important component in session-based recommendation (e.g., people tend to browse product information repeatedly or revisit websites in a period of time). In this paper, we propose a novel model called ReGNN to combine a graph neural network with a repeat-exploration mechanism for better recommendations. Specifically, we dynamically process the item sequence of a session as a graph structure and capture the complex transitions between items by a GNN. Then, we formulate an exact session representation with the attention mechanism. Finally, the repeat-exploration mechanism is incorporated into the ReGNN to model the user’s repeat-exploration behavior patterns and make more accurate predictions. We conduct extensive experiments on two public datasets. The experimental results show that our proposed model ReGNN consistently outperforms other state-of-the-art methods.


I. INTRODUCTION
Recommender systems play an important role in people's daily lives. In many real-world applications, recommender systems help users alleviate the problem of information overload and recommend products that users are likely interested in. Most existing recommender systems are based on user historical interactions or other user information. However, in many cases, a user's profile and past activities are not recorded constantly, which results in only a group of interactions that take place in a user session. To address the challenge above, session-based recommendation is proposed to predict the next item that a user may click or visit when just given the sequence of the user's actions in the current session [1]- [3].
Some conventional methods have been proposed based on Markov chains (MC) [4], [5] to model users' behaviors, The associate editor coordinating the review of this manuscript and approving it for publication was Shangce Gao . which are only based on the previous item that a user visits. The primary problem of these methods is that if a user clicks an item by mistake or randomly, the performance of the model may be strongly influenced. In addition, it is difficult for MC-based methods to model high-order relationships. Later, Hidasi et al. [1] proposed applying a recurrent neural network (RNN) for session-based recommendation and made significant progress. Then, Tan et al. [6] further improved the RNN model by taking data augmentation and distribution into consideration. RNN has natural advantages in modeling user sequence actions. It has proven very useful to capture a user's general interests from a sequence. However, in terms of saving users' long-term behavior information, RNN is not powerful enough, and it models only single-way transitions between session items. Recent years have witnessed significant developments in graph neural networks (GNNs) [7], which have proven useful when learning meaningful representations for graph data. The core idea of GNN is aggregating feature information from local graph nodes using neural networks. Hence, GNN naturally integrates the node information and has been demonstrated to be influential in representation learning [8], [9]. For session-based recommendations, Wu et al. [10] applied GNN to capture complex transitions of items and generate accurate item embedding vectors by modeling the item sequences of a session as a session graph. Based on actual item embedding, GNN learns reliable session representations and predicts the next-click item more precisely.
Although all of the methods mentioned above have made great progress in session-based recommendations, they still have some limitations. None of them addresses the so-called repeat recommendations, which makes up a large proportion of session-based recommendations. For example, people may repeatedly browse product information several times when shopping online, and we tend to listen to the same music frequently in a day [11]. Therefore, Ren et al. [12] proposed a model called RepeatNet, which incorporates a repeat-exploration mechanism into RNN to solve the problem of repeat consumption for session-based recommendations. However, RepeatNet uses GRU to learn the session representation, which is not powerful enough to describe the complex connections of session items.
To adequately capture the complex transitions between session items and accurately model users' repeat-explore behavior patterns, we propose a novel method called ReGNN, which combines GNN and a repeat-explore mechanism to improve the performance of recommender systems. In ReGNN, we process the item sequence that a user interacts with during a session as a session graph, and each item is encoded as graph-structured data. By this method, the repeat behaviors of a user are translated as the edge information of the session graph, which can be accurately learned by GNN to formulate the item representation. Then, we apply an attention mechanism to capture the user's main purpose in a current session, which is combined as a unified session representation. Finally, we employ a repeat-explore mechanism to consider the user's behavior patterns in repeat mode and explore mode and make an accurate prediction. Extensive experiments conducted on two benchmark datasets demonstrate the effectiveness of our proposed model.
In summary, the main contributions of our work are listed as follows: • To the best of our knowledge, this is the first effort that incorporates the user's repeat behavior information into GNN to improve the performance of session-based recommendations.
• ReGNN applies GNN and the attention mechanism to model the complex transactions between items in a session and integrates with the repeat-explore mechanism to adequately capture users' repeated and exploring behavior patterns.
• We conduct extensive experiments on two real-world datasets, which proves the effectiveness of our model and demonstrates that our model significantly outperforms other state-of-the-art methods.

II. RELATED WORK A. TRADITIONAL METHODS
Conventional methods for session-based recommendation include matrix factorization (MF) [13] and Markov chains (MC) [4]. MF factorizes a user-item rating matrix into low-rank matrices; one represents the user latent factors, and the other represents the item latent factors. This method captures the user's general interests based on the user's whole interaction history but ignores the information of the whole interactive sequence. Then, sequential methods based on MC were proposed, which predict a user's next behavior based on the previous behavior. Yap et al. [14] introduced a competence score measure in personalized sequential pattern mining for next-item recommendations. However, the limitation is that it is difficult to capture users' long-term preferences, and the state space becomes unmanageable when including all possible sequence selections [15].

B. DEEP LEARNING-BASED METHODS
Recently, deep neural networks have achieved great success in modeling sequential data. Hidasi et al. [1] applied RNN to model session representation and made great improvements over traditional methods. It learns session representations directly from previous interactions in a session and predicts the next items. Later, Tan et al. [6] further developed an RNN model by using a data augmentation technique to improve the model performance. Afterward, Li et al. [15] proposed a model called NARM with an encoder-decoder architecture for the first time. They employed an attention mechanism on RNN to capture the user's main purpose in the current session, which leads to a promising improvement. Liu et al. [2] proposed a short-term attention/memory priority model (STAMP) to effectively capture the long-term and short-term interests of a session with an attention mechanism. Currently, GNN has been widely used to generate graph-structured data because of its inherent capacity of modeling complex relations between graph nodes. Inspired by word2vec, Perozzi et al. [16] proposed a DeepWalk algorithm to learn node embedding in a graph based on random walks. Following DeepWalk, Tang et al. [17] proposed a network embedding algorithm called LINE. As a modification of GNN, Li et al. [18] proposed a model BPTT to compute gradients by incorporating GRU and GNN. For sessionbased recommendations, Wu et al. [10] proposed a model called SR-GNN, which encodes item sequences as a session graph and incorporates graph models into representing session sequences.

C. REPEAT RECOMMENDATION
Repeat behavior is a common phenomenon in different domains, including web visitation, shopping online, and listening to music. The frequent appearance of the users' repeat behaviors in many scenarios illustrate the importance of repeat recommendations. Many scholars have actively explored this aspect. For example, Ehrenberg [19] studied the repeat buying behavior of customers for consumer bands VOLUME 8, 2020  [11] found that recency is a key factor with a cache-based method, which indicates that users tend to buy items consumed recently. More recently, Benson et al. [20] incorporated time factors, distance factors and quality factors into a holistic model to handle the problem of sequential repeat consumption. Afterward, Ren et al. [12] proposed a repeat-explore mechanism to calculate the score of each candidate item score in two different modes for session-based recommendations.
Most of the session-based recommendation models mentioned above either simply employ GNN to learn the session representation without considering the importance of repeat recommendation. Ren et al. [12] utilize RNN to model the item transitions of a session, which are not powerful enough to capture the complex connection between session items. In a session where has repeated items, the recurrent neural network can not model the relationship between repeated items. However, GNN transforms the repeat information into the edges and learns the item connection to formulate accurate item representations. In this paper, we process the item sequence in a session as a session graph, by which GNN can adequately learn the user's repeat behavior. In addition, we apply a repeat-exploration mechanism to model the user's repeat and explore behavior patterns, which will further improve the performance of ReGNN. To the best of our knowledge, this is the first attempt to incorporate users' repeat behavior information into GNNs for more accurate session-based recommendations.

III. THE PROPOSED METHOD A. TASK DEFINITION
The task of session-based recommendation is predicting the next item that the user will interact with when just given the user's access behavior in the current session (e.g., behavior in a day). Here, we give a formulation of the problem.
Given a user's action session I s = i 1 , i 2 . . . , i j . . . , i n , where i j is the item that the user has visited in a sequence. The goal of our method is to predict the next item that the user will visit in candidate items when given a sequence of the user's previous actions in a session. For example, we predict i n+1 when given I s = i 1 , i 2 . . . , i j . . . , i n . Furthermore, we denote x j ∈ R d as the initial item embedding of i j . As a result, the sequence of a session can also be represented as x 1 , x 2 . . . , x j . . . , x n . We have demonstrated the notation and explanations in Table 1.

B. MODEL FRAMEWORK
The workflow of our model is illustrated in Figure 1. ReGNN is an end-to-end model, and it consists of four layers. In the GNN embedding layer, we first encode the sequence of items in a session as a session graph and initial the item(node) embedding with random normal initializer, each item is a graph node, and some of them have outgoing edges and incoming edges. Then, each graph node is learned by the graph neural network to update the latent vectors of all interacted items. After updating the node embedding several times, we get the final node(item) representations in a session at time t and transmit item vectors to the session generating layer. In the session generating layer, we apply an attention mechanism to evaluate the importance of each item in the session and obtain a global session representation (S g ) by a weighted summation method. In addition, we regard the last interacted item as the user's current interest representation (S c ), and then, we combine S g and S c to obtain the session vector (S re ). In the repeat-explore modeling layer, we divide users' behaviors into repeat patterns and explore patterns. Then, we employ a repeat-explore mechanism to learn repeat mode session representation (S r ) and explore mode session representation (S e ). Finally, in the prediction layer, we calculate the score of visited and unvisited candidate items with different mode representations to obtain the final score of all items.

C. GNN EMBEDDING LAYER
To obtain the latent vectors of graph nodes, we apply graph neural networks to update the initial item embedding. GNN is suitable for item representation learning because it can extract features of session graphs by capturing the transitions between graph nodes. The latent vectors of items are processed by GNN as follows: is the sequence of node vectors with the activated state at time t − 1. The initial state of all node vectors in session I s is x 1 , x 2 . . . x j . . . , x n . W , W z , W r , U z , U r , U are the weight matrices, and b is the bias unit. In addition, σ (·) and tanh (·) are two commonly used activation functions, which refer to a sigmoid function and a tanh function. In addition, is an elementwise multiplication operator, z t j is an update gate and r t j is a reset gate. Finally, E ∈ R n×2n is the edge adjacency matrix of the session graph, as shown in Figure 2. We define E as a concatenation of two matrices E in ∈ R n×n and E out ∈ R n×n , which describe the incoming edges and outgoing edges, respectively. However, the difference from a traditional adjacency matrix is that we define the total edge weights of E in and E out to be 1. For example, in Figure 2, node i 2 has two incoming edges, e 1 and e 4 , and their weights in E in are 1/2. E T is the transposed matrix of E.
The overflow of item vector transition is that for each session graph, e t j captures the connection of all nodes with the state at time t − 1, and then, z t j and r t j determine what information will be updated or reset at time t. Finally, we update all node states and obtain the accurate item representation x t j at time t.

D. SESSION GENERATING LAYER
After learning all the item representations, we further apply an attention mechanism to capture the user's main preference in the current session and formulate the session representation. We denote s g as the global session embedding and s c as the local session embedding. For s c , we denote s c = x t n , where x t n is the last-click item vector at time t. The final session vector and the probability of repeat mode P r and explore mode P e are computed as follows: s re = W re4 s g , s c (8) [P r , P e ] = softmax (W re s re ) where W re1 ∈ R d×1 , W re2 and W re3 ∈ R d×d , W re4 ∈ R d×2d , W re ∈ R d×2 are weight matrices, which control the weights and the latent shape of item embedding vectors. α re j is the weight score of item i j in session I s , which combines the influence of x t j and x t n . In Eq. 7, we employ an attention mechanism to obtain global session representation s g , and then, we concatenate the global session vector s g and local session vector s c in Eq. 8 and process the matrix with a linear transformation to obtain the final session representation s re . Finally, we calculate the probability distribution of the repeat mode and explore the mode by projecting s re into a two-dimensional hidden space and processing it with a softmax function.

E. REPEAT-EXPLORE MECHANISM LAYER 1) REPEAT MODE
To consider the user's repeat behavior pattern, we renewably calculate the session representation in this mode. All parameters are used to learn the repeat behavior characteristics of the user. We calculate the score of items that the user has interacted with in the current session as follows: s r = W r4 s gr , s c (12) y r j = I i j ∈ I s s T r x j (13) VOLUME 8, 2020 where W r1 ∈ R d×1 , W r2 , W r3 ∈ R d×d and W r4 ∈ R d×2d are weight matrices. s gr is the global session vector in repeat mode, and s r is the final session representation, which describes the user's repeat behavior feature in this mode. Finally, we calculate the item score y r j for the items that the user has interacted with in the current session. I (·) is an indicator function. In Eq. 13, the score of an unvisited item will be 0, and we will calculate them in explore mode.

2) EXPLORE MODE
In this mode, we consider the situation in which the user visits new items that have not existed in the current session. The computing process is similar to that in repeat mode, and we calculate the item scores in explore mode as follows:  (17) where W e1 ∈ R d×1 , W e2 , W e3 ∈ R d×d , and W e4 ∈ R d×2d are weight matrices. b e is a bias term. In this mode, all parameters are employed to learn the user's explore behavior characteristics. y e j only calculates the scores for the items that have not appeared in the user's current session, and the scores of previous interacted items will be 0.

F. PREDICTION LAYER AND PARAMETER LEARNING
For recommendation, we combine the item scores from both repeat mode and explore mode. Then, we use a softmax function to normalize the additive item score. y j = softmax(P r y r j + P e y e j ) (18) where P r and P e are the possibility distributions of the two modes. We can also regard them as the weight coefficient of two scores. y j denotes the final item score of item j in the candidate set. For the loss function, we compute the cross-entropy of the item prediction and ground truth in a session as follows: y j log y j + 1 − y j log 1 − y j (19) where y j denotes the ground truth item of a session. In our experiments, we process it as a one-hot encoding vector. For model optimization, we minimize the objective function L y j when training the model, and all parameters are learned in an end-to-end backpropagation algorithm.

IV. EXPERIMENTS
In this section, we will first briefly introduce the datasets and compare the baseline methods and evaluation metrics used in our experiments. Then, we compare ReGNN with other stateof-the-art methods and analyze the importance of different parameters in our model.

A. DATASETS
We conduct our experiments on two real-world datasets as follows.
• Yoochoose is a public dataset from the Recsys Challenge 2015 (http://2015.recsyschallenge.com/ challenge.html), which consists of a stream of user clicks from an e-commerce website for 6 months. Following [15] and [2], we filter out sessions whose length is 1 and items that appear fewer than 5 times.
• Diginetica is a dataset released by CIKM Cup 2016 (http://cikm2016.cs.iupui.edu/cikm-cup). We only use the transaction data in this study. In addition, we also filtered out sessions and items, following the same method of handling the dataset Yoochoose. For a fair comparison, following [10], we generate sequences and responding labels by splitting input sequences. Following [15] and [2], we conduct our experiments on the most recent fractions 1/64 and 1/4 of Yoochoose. The statistics of the datasets are shown in Table 2, and the repeat ratios of the datasets are shown in Table 3.

B. BASELINES
We select the following commonly used state-of-the-art baselines to compare with our proposed model ReGNN.
• POP: POP always recommends the items that have the most frequent occurrence.
• S-POP: S-POP recommends the most popular items of the current session. Ties are broken using global popular values.
• Item-KNN [21]: This baseline predicts the items that are similar to the existing items based on cosine similarity between candidate items.
• BPR-MF [22]: This model optimizes a pairwise ranking objective function and combines it with matrix factorization.
• FPMC [5]: FPMC is a sequential prediction method based on Markov chain for next-basket recommendations.
• GRU4REC [1]: This model employs GRU for session-based recommendations, which utilizes a 98522 VOLUME 8, 2020 session-parallel mini-batch training process and a ranking-based loss function.
• NARM [15]: This is an RNN-based state-of-the-art model to capture the user's main purpose by the attention mechanism.
• STAMP [2]: STAMP applies an attention mechanism to capture users' general interests and current interests.
• RepeatNet [12]: This RNN-based method uses a repeat-exploration mechanism to consider the possibility of user repeat behavior.
• SR-GNN [10]: This baseline transfers an item sequence to a structured graph and uses GNN to learn the session representation.

C. EVALUATION METRICS
We use Precision@20 and Mean Reciprocal Rank@20 to evaluate all compared methods, which are defined as follows. Precision@20(P) is a widely used metric in the SRS area, which represents the proportion of correctly recommended items in the top 20 positions.
Mean Reciprocal Rank@20(MRR) is the average of reciprocal ranks of desired item j. The higher items are ranked in the recommendation list, the higher MRR scores will be. MRR is a normalized score of range [0,1].

D. EXPERIMENTAL SETUP
We set the dimension of item vectors to 100 for all datasets, and the default batch size is also 100. The number of training epochs is 30. Furthermore, we initialize all weight matrices using a random uniform initializer with a standard deviation of 0.1. In addition, the initial learning rate is set to 0.01, and its decay rate is 0.1 every three epochs. All of the experiments are conducted on a GeForce GTX 1080Ti GPU. We have released our experimental code and a small Yoochoose subdataset on GitHub (https://github.com/Jmax302/ReGNN).

E. PERFORMANCE COMPARISONS
To demonstrate the performance of ReGNN, we compare it with other state-of-the-art methods. Our experimental results are illustrated in Table 4. We evaluate the performance of different methods in terms of P@20 and MRR@20. From our experimental results, we make the following observations. First, the most traditional and the simplest algorithms (i.e., POP and S-POP) perform poorly, as they consider only the popularity of items without the user history and personalized customization. Second, the traditional methods Item-KNN and BPR-MF are also not competitive because they cannot learn the information of the user's action sequence. However, Item-KNN performs better than FPMC based on Markov chains. This phenomenon indicates that although FPMC considers the action sequences of a user, it only takes transitions over successive items, which may be problematic in making accurate predictions. For large data, FPMC requires too much memory. That is why the result is missing in Table 4 on the Yoochoose 1/4 dataset. Furthermore, all neural-based methods outperform the conventional algorithms. This proves the effectiveness of deep learning technology in recommendations. GRU4REC applies GRU to capture the interest drift of the user's sequential behavior and achieves a great performance. NARM and STAMP employ the attention mechanism to RNN and make a great improvement. Since RepeatNet uses a repeat-explore mechanism to consider user repeat behavior individually, it performs better than other previous methods. This proves that the repeat behavior of users is an important factor for the next-item recommendation, especially on datasets with high repeat ratios. Different from RNN-based methods, SR-GNN uses GNN to model the connections between items and obtains a good result. This indicates that GNN is superior to RNN on representation learning when given complex connected data.
Finally, our proposed model ReGNN outperforms all the methods mentioned above. When compared with the best baseline SR-GNN, the relative performance improved by ReGNN is 3.76%, 4.51%, 5.84% on P@20 and 12.80%, 14.93%, 18.56% on MRR@20 on the three datasets. The significant experimental results indicate that both parts of the GNN and the repeat-exploration mechanism are important to improve the performance of session-based recommendations. ReGNN not only achieves a great improvement on the relatively high repeat ratio dataset (Yoochoose) but also performs well on Diginetica. This is because on low repeat ratio datasets, the exploration mode of the repeat-exploration mechanism contributes more, and GNN can still learn an accurate session representation.  dataset, we conduct our experiments on Yoochoose 1/64 and Diginetica. Their performance is illustrated in Figure 3. It is obvious that ReGNN performs better with increasing dimension size on both datasets. This is because a high dimension can reflect the potential measurement better. However, when the dimension size is larger than 100, the performance of ReGNN only slightly improves. That is why we set the default dimension size to 100, considering the balance of effectiveness and efficiency.

2) IMPACT OF GNN AND THE REPEAT-EXPLORE MECHANISM
To verify the effect of the different parts in ReGNN, we compare ReGNN with two baselines without considering GNN or the repeat-exploration mechanism. We conducted experiments on Yoochoose 1/64 and Diginetica and show their results in Figure 4.
• ReGNN-g. This model removes the GNN part from ReGNN and replaces it with an RNN to learn the session representation. The repeat-explore mode is reserved.
• ReGNN-re. This model removes the repeat-exploration mechanism and simply calculates the item scores with the session representation learned by GNN. First, the results shown in Figure 4 indicate that ReGNN outperforms both variant models (i.e., ReGNN-g and ReGNN-re) on both datasets. That is, removing the GNN part or the repeat-exploration mechanism will lead to performance loss. Furthermore, ReGNN-g performs slightly worse than ReGNN-re on Yoochoose 1/64 but significantly worse than ReGNN-re on Diginetica. This is because the dataset Yoochoose 1/64 has a relatively higher repeat ratio than Diginetica does. This indicates that the impact of the repeat-exploration mechanism varies with the different repeat ratios of the datasets. The repeat-exploration mechanism can achieve a great performance with frequent repeat behavior of users. In addition, ReGNN-re is relatively stable on both datasets. This shows that GNN can learn the accurate session representation in most scenarios and contributes more to the performance of ReGNN, especially on low repeat ratio datasets. Finally, it is obvious that both parts of the ReGNN are important to improve its performance and make more precise recommendations.

V. CONCLUSION
In this paper, we first studied the problem of the user's repeat-explore behavior patterns in session-based recommendations. Then, we proposed a novel model called ReGNN to obtain the exact representation of a session by adequately modeling the complex relationships between items with GNN and the attention mechanism. Furthermore, we incorporated a repeat-exploration mechanism into ReGNN to model the user's repeat and explore behavior features. Extensive experiments conducted on the two benchmark datasets demonstrate the superiority of ReGNN. In future work, we will incorporate more item features into ReGNN, such as images and item description text, which can describe user preference more accurately, to improve the performance of ReGNN.