SgWalk: Location Recommendation by User Subgraph-Based Graph Embedding

Popularity of Location-based Social Networks (LBSNs) provides an opportunity to collect massive multi-modal datasets that contain geographical information, as well as time and social interactions. Such data is a useful resource for generating personalized location recommendations. Such heterogeneous data can be further extended with notions of trust between users, the popularity of locations, and the expertise of users. Recently the use of Heterogeneous Information Network (HIN) models and graph neural architectures have proven successful for recommendation problems. One limitation of such a solution is capturing the contextual relationships between the nodes in the heterogeneous network. In location recommendation, spatial context is a frequently used consideration such that users prefer to get recommendations within their spatial vicinity. To solve this challenging problem, we propose a novel Heterogeneous Information Network (HIN) embedding technique, SgWalk, which explores the proximity between users and locations and generates location recommendations via subgraph-based node embedding. SgWalk follows four steps: building users subgraphs according to location context, generating random walk sequences over user subgraphs, learning embeddings of nodes in LBSN graph, and generating location recommendations using vector representation of the nodes. SgWalk is differentiated from existing techniques relying on meta-path or bi-partite graphs by means of utilizing the contextual user subgraph. In this way, it is aimed to capture contextual relationships among heterogeneous nodes more effectively. The recommendation accuracy of SgWalk is analyzed through extensive experiments conducted on benchmark datasets in terms of top-n location recommendations. The accuracy evaluation results indicate minimum 23% (@5 recommendation) average improvement in accuracy compared to baseline techniques and the state-of-the-art heterogeneous graph embedding techniques in the literature.


I. INTRODUCTION
The rapid growth of mobile technologies led to the development of online social networks with increasing user counts. As reported in [2], it is expected that it will reach 4.2 billion social media users by the end of 2021. Additionally, the advancements in positioning technology allowed utilizing location data through the social networks in various ways, such as sharing an instant location with other users or exchanging travel experiences with friends [52]. These opportunities foster sharp increases in the use of LBSNs (Location-Based Social Network), such as Foursquare. An LBSN stores users' check-in history, which is a precious resource to analyze Spatio-temporal user behavior The associate editor coordinating the review of this manuscript and approving it for publication was Fabrizio Messina . and to generate recommendations for a variety of aspects such as friends, locations, and activities according to the context of the user [28]. According to [1], datasets provided by Foursquare are used by more than 125,000 developers worldwide. Given an LBSN check-in history (as a set of tuples consisting of user, check-in time, check-in location information) and the user's social network connections, location recommendation for LBSN aims to recommend a set of locations for the target user. As there are too many users and locations in an LBSN, processing and analyzing a formed social network requires considerable effort.
To overcome this difficulty, the use of graph structures and network embedding have gained popularity in recent years with the idea of representing vertices (nodes) in a graph using a low-dimensional vector space that preserves the fundamental structural properties of the graph. Hence, network embedding facilitates identifying nodes having closer characteristics to each other according to their corresponding vector representations. This leads to the adoption of network embedding approaches in various graph analysis tasks by considering node distances (similarities). Recent graph embedding techniques can be classified into two categories depending on the types of input graphs, i.e., homogeneous or heterogeneous graph embedding [14]. In a homogeneous graph, which has nodes of a single type only (such as users in a social network), graph embedding techniques use sequences of nodes obtained via randomly sampled from the graph (using random walks over the graph [38]) for preserving the proximity between node pairs. On the other hand, a heterogeneous graph has a complex structure containing nodes of different types representing different entities and edges linking nodes of the same type or across different entities [16], [19]. For example, an LBSN graph with user (U), location (L), and friend (F) entities contains not only the homogeneous edges between users (friendship) but also heterogeneous edges linking a user and location (visits). These types of graphs are also known as heterogeneous information networks (HINs). Therefore, HINs embody more information, contain richer semantics compared to the homogeneous graphs, and facilitates more complex analysis on heterogeneous data [33].
A heterogeneous graph can be directly embedded by homogeneous graph embedding techniques. Still, it suffers from poor performance in results [14] due to considering different types of nodes as if they are the same. Therefore, network embedding techniques for heterogeneous graphs need to consider information fusing and the structure of the graph while generating node sequences. Many researchers have been working on the techniques for HIN embedding [16], [19], [25], [41], [44], [45], [49], [53], and have proved the usefulness of HIN embedding in HIN analysis tasks. In such studies, some of the techniques rely on meta-paths in a random walk to generate node sequences for preserving the similarity between nodes [16], [19], [41]. Some other techniques decompose the HIN into simple networks (bi-partite graphs) and then jointly learn the embeddings of each sub-network [13], [44], [53]. Additionally, there are solutions that employ neural network-based approaches to learn non-linear mapping functions for HIN embedding [25], [49], [50], [56]. HIN's embedding learned by the aforementioned techniques has been used in many tasks, such as classification [26], clustering [30], link prediction [31], and recommendation [33], [59].
A critical limitation in the existing HIN embedding methods is that they lack a clear mechanism to cope with the spatial context in a recommendation. In location recommendation, spatial context is a frequently used consideration such that users prefer to get recommendations within their spatial vicinity. Therefore it is needed to capture the complex relations between different types of entities locally with respect to the spatial regions (spatial contexts). To be able to cope with this problem, in this paper, we propose a new heterogeneous graph embedding technique SgWalk (Subgraph Walk), that decomposes the graph into smaller subgraphs constructed for a user with respect to spatial context of the user, which is considered as the vicinity of the user (the region with a given radius around the user's current position). In the constructed user subgraphs, we consider a wide range of information sources such as personal, social, spatial, and trustworthiness contexts. Inclusion of trust relationship is also novel for constructing heterogeneous networks for LBSNs [52]. Location context-based user subgraph, which is a HIN including a variety of entities and the node embeddings constructed by the proposed method, is utilized in node to generate more accurate recommendations for the given user. The SgWalk technique consists of a four-step process that explores the proximity between users and locations and generates recommendations. In the first step, it builds user subgraphs according to the spatial contexts, and as the second graph, it generates random walk sequences over the user subgraphs. In the third step, it learns embeddings of nodes in the LBSN graph and finally generates location recommendations for the given user.
The major differences of the proposed node embedding technique are utilizing contextual user subgraphs instead of bi-partite subgraphs proposed in the literature and not depending on meta-paths. We can highlight the contributions of the study as follows: • We propose a user subgraph-based graph embedding technique SgWalk. Instead of a global LBSN graph, we use subgraphs to capture the relations among heterogeneous entities within spatial contexts.
• We adopt the popular word embedding technique by applying the random walk sampling process on the user subgraph as a new setting for heterogeneous information network embeddings.
• We work on heterogeneous information networks that consist of trustworthiness, user preferences, social connections, and spatial-context notions and fuse them effectively in learning node embeddings. Modeling trustworthiness as the trusted user is also a novelty of the study. We analyze the effect of trusted user nodes on the location recommendation performance through ablation studies.
• In order to analyze the recommendation accuracy of the proposed technique, we conduct extensive experiments on four real-world datasets.
The rest of the paper is organized as follows: Section II reviews the related studies in location recommendation and graph embedding. We present preliminaries and the details of SgWalk in Section III. The experiments and the analysis results are given and discussed in Section IV. The paper is concluded in Section V with final remarks.

II. RELATED WORK
In this section, we give a brief overview of previous work addressing recommendation in location recommendation and network embedding techniques. VOLUME 9, 2021

A. LOCATION RECOMMENDATION
Early studies on location recommendation analyzed GPS trajectory logs collected from users of social networking services [12], [60], [61]. However, recent studies employ check-in data in LBSNs due to its availability [5], [11], [51], [55]. There are five major information categories basically used in location recommendation studies: (i) user preferences, to recommend attractive locations matching up attributes of a user profile, (ii) social relationships, generating recommendations from the check-in histories of friends who are expected to have similar interests, (iii) spatial-context, clustering locations according to their geographic positions to improve recommendation accuracy, (iv) the temporal effect, utilizing the idea that human activity has time periodicity such as daily or weekly, (v) trust-awareness, relying on the implicit or explicit trust networks of the users.
There is rich literature considering these five categories in the recommender systems. Zheng et al. [60] converted GPS trajectory data to factorized location-activity matrix to recommend locations for a given activity, or vice versa. Cho et al. [12] investigated human mobility patterns on location-based social networks through temporally and geographically periodic movements with the social network structure. Montjoye et al. [15] showed that human activity has a substantial time periodicity by confirming that users' check-in patterns may be cycled at one day or one week. Gao et al. [21] studied the temporal cyclic patterns of user check-ins in terms of temporal consecutiveness. Temporal cyclic patterns are also employed by Wang et al. [51] together with the user's check-in time and the corresponding check-in location to improve the recommendation accuracy.
Focusing on spatial information, Lian et al. [27] proposed a weighted matrix factorization-based location recommendation approach to integrate geographical influence into matrix factorization by using users' and locations' latent feature vectors. Wang et al. [49] introduced a Collaborative Deep Learning (CDL) approach, which extracts deep feature representations from the content and captures the similarity and implicit relationship between locations (and users) by performing deep learning collaboratively. The learned representations are used for location recommendations. Yin et al. [56] improved [49] by adding geographical information and proposed a Spatial-Aware Hierarchical Collaborative Deep Learning model (SH-CDL). This model jointly performs deep representation learning for locations from heterogeneous features and hierarchically additive representation learning for spatial-aware personal preferences.
Since the trust concept is one of the entities considered in the graph structure used in our study, we present a glance at how trust is used in recommendation studies. Trust is one of the recent and popular concepts employed in graph-related problems and recommendation methods for improving accuracy. In a social network, when trust between the users is explicitly available, this information is modeled as an unsigned trust network. When explicit distrust is provided as well as explicit trust, such relationships are considered as a signed trust network. Additionally, there are efforts to model implicit trust in order to improve inference on social network [35]. Trust notion has been studied within the scope of a variety of tasks such as influence diffusion modeling [48], and answer selection [43]. For recommendation problems, trust modeling and trust networks are generally used for alleviating the matrix sparsity through auxiliary trust information [3], [4], [54]. To the best of our knowledge, trust modeling has not been incorporated into graph structure for location recommendation in the previous studies.
The summary of location recommendation methods given in this section presents an overview of the employed techniques and how they evolved from matrix factorization towards deep neural architectures. As seen in the summaries, although they focus on the same problem, the settings considered in these studies are different than our approach. One important observation is that the trust concept is not well studied in location recommendations. Considering its use in the other recommendation tasks, it promises the potential for improvement for location recommendations.

B. NETWORK EMBEDDING
Network embedding aims to find a low-dimensional vector representation of the network or its components by preserving the network structure, and properties of the nodes in the network [7], [14], [58]. In the early studies, [36], [40], dimension reduction techniques decomposed the network for learning the latent low-dimensional vector representations of nodes and edges.
Lately, deep learning-based models have taken place in network embedding to handle the decomposition issues in large networks. Motivated from Word2Vec [34] approach, Perozzi et al. [38] proposed Deepwalk that fed the generated random walk sequences to the Skip-gram model to learn network representations. DeepWalk assumed random walk sequences as sentences and nodes as words and then maximized the co-occurrence probability of each node pair. Next, node2vec is proposed by Grover et al. [22], in which a biased random walk is performed on homogeneous networks by employing breadth-first and depth-first sampling. With the LINE model, Tang et al. [45] employed both first and second-order proximities between nodes by edge-sampling for learning vectorial representations of nodes in large information networks. Furthermore, higher-order graph proximity for network representations is put forward in the GraRep model by Cao et al. [9]. Unfortunately, these state-of-the-art methods concentrated on learning the representation of nodes in a homogeneous network and cannot be directly applied to heterogeneous networks.
Recently, heterogeneous information networks have gained popularity due to their capability to model nodes representing several different entities and specific relations. HIN embedding aims to represent all different types of nodes into the shared low-dimensional space. Actually, HIN embedding methods improved the ideas in homogeneous graph embedding research to handle the heterogeneity of the information graph. For example, Metapath2vec [16] as an extension of Deepwalk, performs meta-path based random walks to generate node sequences for accommodating heterogeneous neighborhoods of a node and apply the Skip-gram model to learn the representation of heterogeneous networks. HIN2Vec [19] discovers different types of relationships among nodes by combining meta-paths shorter than a certain length and employs them to generate better walk sequences. JUST [24] performs random walks on the HIN by applying a jump and stay strategy to control the walks over the heterogeneous nodes without depending on a meta-path. On the other hand, PTE [44] extends LINE by decomposing the heterogeneous graphs into bi-partite subgraphs and performs individual network embedding by using the LINE approach. GE [53] and JLGE [13] also extend the LINE approach, but for embedding location nodes. Both GE and JLGE decompose LBSN into bi-partite user-location subgraphs and then jointly perform representation learning on node pairs in subgraphs.
Lastly, deep neural network-based methods are imported to heterogeneous data embedding methods due to the power of deep models, such as autoencoder and graph neural networks. SHINE [50] borrows the autoencoder model to encode and decode the heterogeneous information in the social network to obtain the feature representation, and also in [37], [56], the stacked denoising autoencoder (SDAE) is used as the deep learning model for feature representation. Graph Neural Network (GNN) is another model which learns the graph representations using specially designed neural layers and defines convolutions in the graph domain by aggregating feature information of each node from the connected neighbors. GraphSAGE [23] is proposed as the seminal spatial-based GNN framework that is founded upon the general notion of aggregator functions for the node embeddings. Zhang et al. [57] proposed a heterogeneous GNN model to consider heterogeneous graph structure and the heterogeneous contents together by collecting strongly correlated heterogeneous neighbors. Also, for better representation of heterogeneous graphs with rich node content features, in the MAGNN method, Fu et al. [20] employs intra-meta-path aggregation for the content transformation of heterogeneous node attributes. Furthermore, Duran et al. [17] proposed a method based on Graph Convolutional Networks (GCN), which benefits from N-partite graphs to represent N-multiple user interaction contexts to learn node embeddings and addressed on recommender system for seamless integration of user contexts.
Our proposed location recommendation method focuses on heterogeneous LBSN graphs and offers a subgraph-based node embedding technique. In this sense, it has similarities with the HIN embedding methods in the literature. Due to this, in the experiments, we provide a comparison with a variety of such methods such as metapath2vec, HIN2Vec. GE and JLGE are also relevant to our work since they also challenge location recommendation problems and provide a network embedding-based method. Therefore we included JLGE in our analysis for comparison. For the GNN based methods, most of the summarized techniques work on different graph structures than our study, mostly lacking either social network or location interaction. Among them, MAGNN is the most compatible one with our study. Hence it is considered for comparison. In the literature, only a few research efforts can be found that employ subgraphs in the embedding process for recommending locations to the users. Our approach differs from the previous works in the subgraph creation process. In the aforementioned studies, graphs are decomposed along with relationships or meta-paths. However, we decompose the graphs according to the geographical attributes of the nodes such that location nodes around the user's current positions with the specified radius and user nodes (friend, expert, trusted user) visiting these locations. With this approach, we aim to model users' preferences and interactions with respect to spatial context.

III. HETEROGENEOUS GRAPH EMBEDDING WITH SgWalk
In this section, we introduce our method SgWalk, a heterogeneous graph embedding method using random walk on user subgraphs rather than using the whole LBSN graph. Basically, we develop a graph embedding-based solution combining random walks with a SkipGram-like model, which has been widely adopted in the literature and provides high accuracy performance on different tasks [16], [19], [22], [24], [38]. Our proposed method first constructs user subgraphs with respect to spatial contexts. This provides focusing on user preferences and relationships between users and locations for the spatial context and helps to eliminate unintended users and uninterested locations. Then random walk is applied over generated heterogeneous input graphs to obtain walk sequences to feed into the SkipGram model for learning the node embedding. An illustration of these steps is given as an overview in Figure 1. In the rest of the section, we present the details of user subgraph construction strategy, node embedding learning process using SkipGram, and recommendation generation, after problem definition and preliminaries.

A. PRELIMINARIES
We can define personalized location recommendation problem as follows: Given a graph G corresponding to a particular LBSN, a user u ∈ U and the current location of the user, we aim to generate a set of locations {r l 1 , . . . , r l i , . . . , r l n } that have not been visited by u as recommendations, such that each r l i ∈ L in the vicinity. The challenge is to populate this recommendation set with the highest accuracy. We follow four steps to fulfill this challenge: user subgraph construction, random walk generation, graph embedding, and location recommendation. Main notations used in this paper are given in Table 1. In the paper, capital letters are used for representing sets; small letters are used for constants value, vectors and matrix symbols are displayed with indices.

Definition 1 (Location):
A location is a specific site (e.g., a cafe or a movie theater) that can be uniquely identified. In our dataset, a location has two attributes: identifier (name), geographical location. We use l to represent a location and l c to denote its corresponding geographical coordinates in terms of longitude and latitude.
Definition 2 (Check-in Activity): A check-in activity is expressed as a quadruple a(u, l, l c , τ ) that denotes that in activity a, user u visits the location l on coordinates l c at time τ .
Definition 3 (Vicinity): Vicinity is a circular region defined by a user's current location p u and the radius parameter ρ such that | p u − l c | < ρ. Obtained circular area is used as the recommendation region.
On the basis of the definitions for the basic LBSN concepts, we formally define the problem that we focus on in this study as follows.
Problem 4 (Location Recommendation): Given a check-in activity set A = {a 1 ,a 2 ,. . . ,a n }, a querying user u q in U within a vicinity, our goal is to find latent representations of v in V and generate a set of Top-N locations { l 1 , . . . , l N } as recommendations, so that the recommendation set consists of previously not visited locations u q will be visited next time with the highest accuracy.
In this work, we devise a user subgraph-based location recommender, and we hypothesize that utilizing user subgraphs of spatial context to generate random walk sequences improves the recommendation accuracy.

B. USER SUBGRAPH CONSTRUCTION
We employ an undirected and unweighted graph model to represent a particular subgraph. This graph, G, is a tuple G < V, E > where V is a set of nodes v and E is a set of edges e. V ⊂ (U ∪ L) where U is a set of users and L is a set of locations. Note that V contains different subsets of U and L. Moreover, it includes six different types of nodes: user, friend, expert, trusted user, location, and popular location. Node types in a subgraph are formally defined as follows: Definition 5 (Friend): Each user who takes part in an explicit direct relationship (friendship, following, etc.) in the social networks is considered as friend. Friendship relationship between two users u i and u j denoted as a tuple of (u i , u j ).
Definition 6 (Trusted User): A user is considered as trusted if the user's check-in activity behavior conforms to common behavior in visiting locations in a particular region.
Trustworthiness score calculation of a user is done according to the formulas stated in [8], which is adapted from the trust model developed in [35].
Definition 7 (Expert): LBSN data typically do not include expert information explicitly. In this work, we employ a HITS-based [10] algorithm to determine the experts in the vicinity. In the HITS-based method, people who visit many high-quality locations in a region are regarded to have rich knowledge about the locations in the region. In the algorithm, users' expert scores are calculated by iteratively applying the formula in Equation 1. The users are sorted in decreasing order with respect to the score, and the top n% of the users are considered as experts.
Definition 8 (Popular Location): As in expert information, LBSN datasets do not include any explicit information about the popularity of the locations. So we employ the same HITS-based [10] algorithm to find the popular locations in the vicinity, as well. In the algorithm, if a location is visited by many experts, it is more likely to be a popular location [6]. Location scores are calculated by iteratively applying the formula in Equation 2. The locations are sorted in decreasing order with respect to the score, and the top n% of the locations are considered as popular locations.
Expert user and popular location score calculation algorithms run simultaneously in an iterative manner as expressed in Equation 1 and Equation 2. In the equations, R u i denotes the expert score of user u i , R l j denote the popularity score of the location l j and A u i , A l j denotes the check-in activities of user u i or location l j respectively.
In each iteration, popularity scores are updated by using the previous iteration's expert scores. In the same iteration, newly obtained popularity scores are employed for updating the new expert scores. At the end of each iteration, popularity scores and expert scores are normalized. The algorithm runs until the values converge.
User subgraph is constructed incrementally according to the types of nodes. The simplest user subgraph that contains user, location, and friend nodes is constructed by depending on their own and friends' check-in history (Figure 2a). It is further enriched by adding trusted users and popular locations visited by them (Figure 2b). In a similar way, experts and their visited popular locations are also included ( Figure 2c). Finally, a complex user subgraph is constructed (Figure 2d) by using all introduced nodes, which are user, friend, location, expert, trusted user, and popular locations. In order to analyze the effect of including such additional types of nodes, we conduct accuracy performance analysis experiments by using each subgraph structure separately.
In the user subgraph construction, four different contexts, such as personal, social, trustworthiness, and location (spatial) contexts, are considered to obtain an improved recommendation results. The details of user subgraph construction are given in Algorithm 1. In this algorithm, userid and cur-rentLoc represent the target user for recommendation and the user's current location for vicinity, respectively. In the algorithm, GetUserLocationsInVicinity procedure retrieves the previous check-ins of the current user in this region. Similarly, GetFriendLocationsInVicinity procedure gets the check-in history of the friends in this region. Likewise, GetExpertLocationsInVicinity and GetTULocationsInVicinity procedures find the check-ins of popular locations visited by experts and trusted users in the recommendation region, respectively. Following this, friends of the target user who have check-ins in the vicinity are retrieved in the GetFriendsOfUsermethod. In GetExpertsInVicinity and Get-TrustedUsersInVicinity methods, Top-N experts and trusted users having check-ins in the vicinity are fetched. Once all those users are collected, the relationships between the current user and these users (i.e., friends, experts, and trusted users) are established. Finally, the visited location nodes and the edges between users and these locations are added to the subgraph. In the algorithm, trustedUserCount, expertCount, and popLocCount parameters limit the number of trusted users, experts, and popular locations in the subgraph.

C. RANDOM WALK WITH SgWalk
We perform random walks on the constructed user subgraphs to sample from input heterogeneous information networks. Formally, for each user node u in U, we initialize a random walk sequence starting from u until the walk length is reached. Similar to existing random walk based graph embedding techniques [16], [19], [24], [38], we generate a set of node sequences with length s by performing a specified number of random walks r on the subgraphs of each node u in U. Algorithm 2 illustrates our random walk process to generate a set of walk sequences.
User subgraph constructed by Algorithm 1 is used within Algorithm 2. In this algorithm, walkLength and walkCount variables denote the maximum walk length and the number of sequences per subgraph, respectively. For each user, the spatial context on which the latent representation is to be constructed through random walks s is determined through FindClusters method. This method applies clustering algorithm DBSCAN [18] on the check-in activities A u of user u with the radius parameter ρ. We prefer this algorithm since trustedUserCount ← the value denoting the top-n number of trusted users in the vicinity 3: expertCount ← the value denoting the top-n number of experts in the vicinity 4: popLocCount ← the value denoting the top-n number of popular locations in the vicinity 5: G < V , E > ← ∅ 6: vicinity ← GetUserLocationsInVicinity (userid, currentLoc) 7: vicinity ← vicinity ∪ GetFriendLocationsInVicinity (currentLoc) 8: vicinity ← vicinity ∪ GetExpertLocationsInVicinity (currentLoc, popLocCount) 9: vicinity ← vicinity ∪ GetTULocationsInVicinity (currentLoc, popLocCount) 10: users ← GetFriendsOfUser (userid) 11: users ← users ∪ GetExpertsInVicinity(currentLoc, expertCount) 12: users ← users ∪ GetTrustedUsersInVicinity(currentLoc, trustedUserCount) 13: for all user in users do 14: E ← E ∪ new edge between currentuser and user 15: end for 16: V ← currentuser 17: V ← V ∪ users 18: for all data in vicinity do 19: V ← V ∪ new location vertex for data.location 20: E ← E ∪ new edge between data.user and data.location 21: end for return G < V , E > 22: end procedure DBSCAN is based on neighborhood and density concepts, and it does not require the number of clusters in advance. While random walks are generated on the subgraphs, at each move, nextLocation and nextUser variables store the location and the user nodes currently being visited by the random walk algorithm, respectively. In each iteration, nextLocation changes, and the visited node id is appended to the walk sequence. When the random walk length is reached, the algorithm returns the generated random walk sequence. Since our LBSN graph has unweighted and undirected edges from user to location, the transition probabilities are equal for all the nodes in every random walk step. Therefore, the next node is selected by uniformly sampling a node from the neighbors of the current node in both directions.

D. NODE EMBEDDING LEARNING WITH SkipGram
Since the frequency distribution of vertices in random walks of social network and words in a language both follow a power-law [39], in our method, we use a technique similar to Word2Vec [34] to generate the low dimensional vectorspace (d-dimensional) representation of a node in the graph. We adopt a Skip-Gram model with hierarchical softmax to generate the node embeddings. SkipGram is a model that maximizes the co-occurrence probability among the nodes that appear within a window with size w in the set of walk sequences, and hierarchical softmax is utilized to speed up the training phase. Formally, for a pair of nodes, v i and v j appearing in the window in the set of walks W, the co-occurrence probability is defined as given in Equation 3.
In the equation (·), is the sigmoid function (x) = 1/(1 + exp(−x)), and v i and v j refer to the embeddings (vectors) of v i and v j , respectively. In addition, the Skip-Gram model employs negative edge sampling techniques to improve embedding accuracy by a randomly sampled edge between negative nodeṽ k and v i that is not appearing in the set of walks. The negative edge probability is defined as given in Equation 4.
Here negative samples (nodes) are uniformly drawn from node distributions in the walks. In summary, for the pair of nodes (v i , v j ), the SkipGram model maximizes the objective function as given in Equation 5.
In the equation, N is the number of negative edge samples. Parallel Asynchronous Stochastic Gradient Descent (ASGD) is utilized to learn the node embeddings efficiently by iterating over all node pairs appearing within a context window of walkLength ← maximum number of node in a random walk sequence 3: walkCount ← count of random walks sequences per user subgraph 4: walks ← ∅ 5: for all user in U do 6: userLocations ← FindClusters(user) 7: for all curLocation in userLocations do 8: G < V , E >← GenerateUserSubgraph(user, curLocation) 9: nextLocation ← null 10: while i < walkCount do 11: walkSequence ← ∅ 12: while len(walkSequence) < walkLength do 13: nextUser ← SelectNextUser(nextLocation) 14: walkSequence.append(nextUser) 15: nextLocation ← SelectNextLocation(nextUser) 16: walkSequence.append(nextLocation) 17: end while 18: walks.append(walkSequence) 19: i ← i + 1 20: end while 21: end for 22: end for return walks returns the set of generated random walk sequences 23: end procedure size w in each walk sequence. The optimal value for dimension size, d, of embedding vectors and the window size, w, (i.e., the context size) for model training using the Skip-gram model are determined through experiments whose details are given in the Experiments section (see Section IV-E).

E. LOCATION RECOMMENDATION
Once the graph embedding of the nodes in LBSN is performed, the vectorial representations of locations and users are learned in a shared space. For a given user u ∈ U, we calculate the prediction score for each of the unvisited locations l j ∈ L in the vicinity according to the Equation 6, and then we recommend top N of the ranked list of locations having the highest scores.
In the equation u and l j refer to the embeddings (vectors) of u and l j , respectively.

IV. EXPERIMENTS
In this section, we present the experiments conducted for empirical evaluation of our approach, SgWalk. We first describe the datasets used in the experiments and evaluation metrics. Then evaluation strategy and parameter setting used in the analysis are presented. In the experiments, we firstly present an analysis on the effect of including different node types in the subgraph for the recommendation accuracy. Then we give the evaluation results on the impact of two key parameters, vector dimension and window size, on the quality of the learned embeddings. Then, we compare our proposed technique against the state-of-art methods. Statistical significance of the comparative results is presented as well. Lastly, validity threats and limitations of the proposed method are discussed.

A. DATASETS
For the evaluation, we use four real-world datasets involving heterogeneous graphs, which are Brightkite [12], Foursquare [55], Gowalla [12] and Wee places [29] datasets. All of the datasets contain users' check-in data for the locations and the friends of users. In the experiments, we use the subset that is filtered for New York City. The main characteristics of used datasets are summarized in Table 2. As shown in the table, although these datasets have close average check-in counts per user and the average number of friends per, Foursquare has nearly double the number of user, location, check-in, and friendship counts compared to Brightkite. In the datasets, some of the users tend to visit the same location more than once. This hinders the evaluation since recommending new locations to such users can never provide a hit count. To eliminate this problem, we simply grouped the records by user and location. In particular, for each user, we use the latest check-in for a location if more than one check-in exists and ignore the others. The user data is sorted by check-in date from the oldest to recent and partitioned into training and test datasets.

B. EVALUATION METRICS
We utilized three widely used metrics to evaluate the accuracy of SgWalk and baseline methods, which are precision@k, recall@k, and f-measure@k [5], [14], [19], [33]. Precison@k is calculated by checking whether locations in the ground truth are ranked in the recommendation list of size k. It is calculated as given in Equation 7. precision@k = number of true locations in recommendation@k number of recommendations@k (7) Recall@k measures the ratio of truly recommended locations to the all visited locations in the ground truth, which is calculated as given in Equation 8. recall@k = number of true locations in recommendation@k number of locations in ground truth (8) Precison@k and Recall@k are inversely affected in evaluations. In other words, when Precison@k is high, Recall@k is low or vice-versa. To normalize these metrics, f-measure is used, which is calculated as given in Equation 9.
In the experiments, for all of the metrics, the accuracy is analyzed under k values of 3, 5, 8, 10, 15, 20.

C. EVALUATION METHODOLOGY AND PARAMETER SETTINGS
Since the challenged recommendation problem and the proposed solution involve spatial context, we need to acquire locations for the test users. To be able to simulate the current position of the user and locations in the vicinity, we cluster the check-in data with respect to locations. As the clustering algorithm, we use DBSCAN [18]. We prefer this algorithm since it does not require the number of clusters in advance, and it determines clusters based on the concepts of neighborhood and density. To fulfill this, DBSCAN needs two parameters: the radius of the neighborhood and the minimum number of points in a neighborhood. In the experiments, we set the minimum number of points as 3 and neighborhood radius as 2000m which is also consistent with the optimal neighborhood size in [32].
Once the clustering is performed, the center of each cluster is considered as the current location of the user to define the spatial context of the user and the vicinity for the recommendation. The clustered user data is sorted by check-in date from old to recent. The sorted check-in data for each cluster are partitioned into training and test datasets such that the old data constitute the training dataset since we aim to find the next location recommendations. Data is partitioned such that 80% of the data per user is used for training, 20% is for testing. The experiments are conducted under 5-fold crossvalidation. We set the walk to count r to 10 per subgraph and the length of walk sequence s to 40 in random walk generation processes.

D. ANALYSIS ON THE EFFECT OF ADDING DIFFERENT NODE TYPES
As the first analysis, we compare the recommendation accuracy under the subgraphs A, B, C and, D illustrated in Figure 2a, 2b, 2c, and 2d, respectively, to assess the impact of adding new type of node. The results of the experiments performed on Brightkite, Foursquare, Gowalla, and Wee Places datasets are given in Table 3. The results clearly indicate that types of nodes affect the accuracy of recommendation in terms of precision, recall, and f-measure metrics. Friendship relation exists on all datasets, so we set it as the baseline subgraph. Then we investigate the impacts of 2 latent node types, Expert and Trusted Users. As expected, the subgraph having only friend nodes and its check-ins (subgraph A) has the lowest recommendation accuracy in the experiments among all subgraphs. It can be concluded that the friends of a user cannot cover enough number of location recommendations for the user's current context. Experiment results for subgraph B and subgraph C indicate that all subgraph acquire higher accuracy values than subgraph A. In addition, subgraph B performs better than subgraph C on all datasets. It is reasonable since if a location is visited by an expert, then it has a higher possibility to be visited by the other users. On the other hand, trusted users may recommend essential places, so its contribution is not as much as expert. As the last option, we consider the subgraph containing all node types together (subgraph D), and then we come up with the best accuracy in all datasets. Experts and trusted users have different check-in behavior and including these latent nodes improves the accuracy from popularity and trust-worthiness aspects, respectively. For example, the fmeasure value is improved by 58% @5 recommendation in the Wee Places dataset. These results also indicate that combining disjoint types (friend, expert, trusted user) enhances the accuracy of the recommended locations.
In all datasets, we observe a similar accuracy pattern. The amount of accuracy gap varies, but the increase in the accuracy with the inclusion of different types of nodes is consistent for all datasets. It is an important observation revealing that the accuracy of SgWalk is not dataset-specific. Although these four datasets' characteristics differ in terms of Check-ins per User and Friends per User values, Subgraph D, which contains friend, trusted user, and expert nodes together, attain the best accuracy in all test cases. Furthermore,  experiment results are more distinctive for a small number of recommendations, but all subgraphs yield similar results by increasing the recommendation count. Subgraph D has its best f-measure value with a recommendation count of 3 for Brightkite and Wee Places and also 5 for Foursquare and Gowalla, and performance of top 5 in Brightkite and Wee Places is closer to top 3 than top 8. So we use the top 5 as the recommendation count in the following experiments.

E. PARAMETER TUNING EXPERIMENTS
In this section, we present the results of tuning the parameters of our embedding model described in Section III-D. In particular, we examine the impact of window size w and the embedding vector dimension d when we train the model using Skip-gram language model. The candidate values of these two variables are w = 1, 2, 3, 4, 5, 6, 7, 8 and d = 100, 120, 150, 200, 250, respectively.
Here we examine the different values of the window size w and the dimension size d, i.e. (w, d) pairs, that provide recommendation accuracy in terms of f-measure. Table 4 shows SgWalk's f-measure values for the top 5 predictions for different values of w and d.
It is noticeable that as the dimension d increases until around 200, the accuracy increases as well. Increasing the dimension of the vector has an inverse effect after the convergence point. Therefore, we set the dimension size to 200 for all datasets.
On the other hand, we observe that the window size w has an interesting effect on the f-measure value. It converges alternatingly according to whether w is odd or even ( Figure 3). This is due to the nature of the generated walk sequences, which contain user and location nodes alternatingly. F-measure has the highest value at w = 2 and drops and increases gradually as long as the window size w increases. Having lower w values also reduces the computational cost significantly. In Table 4, we highlight the parameter values, which give the best accuracy and are used for the rest of the evaluation.
In the parameter tuning experiments, the ranges of the parameter values are determined manually. After the experiment runs are completed, we examine the accuracy results and check if the maximum value is obtained by the boundary values of the parameters. If so, we conduct new tests by increasing or decreasing the value of the parameter according to the boundary condition (being either the min or max value in the previous experiment run). We continue until the parameter value that provides the maximum accuracy falls within the minimum and the maximum values (i.e., not a boundary value). For example, for vector dimension size tuning, we started with dimensions 100, 120, and 150, but the maximum value was obtained at the boundary of the dimension size values. For this reason, we conducted new tests with dimension 200. Nevertheless, the maximum value was still obtained at the boundary. And then, we executed new tests with dimension 250. Finally, the dimension size setting providing the maximum accuracy fell inside the minimum and maximum boundaries. Therefore we concluded that the optimal dimension value is 200 for the SgWalk method.

F. COMPARATIVE RECOMMENDATION ACCURACY ANALYSIS
We analyze the accuracy performance of SgWalk under varying recommendation counts against the state-of-the-art graph embedding methods in the literature. HIN embedding is obtained in the literature in several ways, such as using homogeneous embedding techniques, employing meta-paths, bi-partite decomposition or using deep learning techniques. We select the baselines from each of these categories and also consider the techniques that use social relations. In the experimental setups, we use the default values for the parameters if it is not specified otherwise in the original work. The details of the compared methods are as follows: • DeepWalk [38] learns node embeddings by first performing classical random walks on an input graph and then feeds the generated random walks to a SkipGram model. DeepWalk was originally designed for homogeneous graphs, so it is applied to a heterogeneous graph by ignoring the heterogeneity and treating all nodes and edges of the graph are in the same type. We set the number of walks per node r = 10, length of walk sequence s = 40, and the window size the w = 10 and embedding vector dimension d = 100 for SkipGram.
• PTE [44] is a semi-supervised model for learning text embeddings using both labeled and unlabeled data.
In these experiments, we used PTE in an unsupervised way. Specifically, for a heterogeneous graph, we created bipartite subgraphs as follows: (U)ser-(L)ocation, (U)ser-(F)riend, (U)ser-(T)rusted, (U)ser-(E)xpert, and then fed these graphs to PTE to generate the node embeddings.    for the parameters r, s, w, and d as in the DeepWalk experiments.
• HIN2Vec [19] combines a set of meta-paths shorter than a certain length to perform meta-path guided random walks in order to learn both node embeddings and meta-path embeddings jointly. In these experiments, we set the maximum meta-path length to 3 and pro- • JUST [24] is a heterogeneous graph embedding technique using random walks with the jump and stay strategies to learn node embeddings more efficiently.
In these experiments, we set the stay or jump parameter across domains α as 0.5 and provide five domains as follows: (U)ser, (L)ocation, (F)riend, (T)rusted User, and (E)xpert User with the allowed transitions between domains. For random walk generation and the SkipGram process, we used the same values for the parameters r, s, w, and d as in the DeepWalk experiments.
• JLGE [13] is a spatial-temporal graph-based model that recommends location with learnt embeddings. It extends VOLUME 9, 2021 The results clearly indicate that SgWalk consistently performs better than the compared methods in terms of precision, recall, and f-measure metrics, especially for recommendation count 8 and below. For the higher number of recommendations (10, 15, and 20), for the majority of the results, we observe higher accuracy results by SgWalk. As a summary, accuracy improvement with respect to the compared methods (in terms of f-measure under @5 recommendations) across the datasets are given in Table 5. It is observed that SgWalk provides the maximum improvement against DeepWalk with 102% on average. The minimum improvement is over JLGE with 23% on average. These results show a strong indication that SgWalk, focusing on spatial context and generating node embeddings by using subgraphs with respect to spatial context, can capture the contextual relationships more effectively. JLGE has the second-best accuracy value in the evaluation. However, JUST provides higher recommendation accuracy than JLGE and PTE on the Brightkite dataset, and MAGNN has better results under a high number of recommendations on Foursquare and Gowalla. JLGE, which is not a random walk-based approach and designed for location recommendation, on average, performs better than other random walk-based approaches JUST, metap-ath2vec, HIN2Vec, and DeepWalk. Among the meta-path based approaches MAGNN, metapath2vec and HIN2Vec, MAGNN is the best with the help of deep learning techniques, and metapath2vec performs better than HIN2Vec under a fewer number of recommendations. On the other hand, HIN2Vec first catches and then beats metapath2vec as the recommendation count increases. DeepWalk has the lowest accuracy since it is originally designed for homogeneous graphs.
The accuracy of the techniques also varies depending on the characteristics of the datasets (given in Table 2). However, on all datasets, SgWalk attains the best accuracy values (for all f-measure values @10 or below, and the majority of f-measure values @15 or above). Hence, we can conclude that the accuracy of SgWalk is not dataset dependent. When we analyze the accuracy of the state-of-the-art methods on datasets, we see that DeepWalk, HIN2VEC, and MAGNN perform better on the Foursquare dataset, PTE, and metapath2vec perform better on the Gowalla dataset, JUST and JLGE perform better on the Brightkite and Wee Places datasets, respectively. This shows the characteristics of the datasets have an effect on the methods. The results also show that Check-ins per User value affects the recommendation accuracy. The recommendation accuracy increases as the number of Check-ins per User increases. Moreover, when the number of recommendations is low, accuracy results are more distinctive, but with the increase in the number of recommendations, all techniques produce similar results.

G. STATISTICAL SIGNIFICANCE ANALYSIS FOR COMPARATIVE EVALUATION RESULTS
Statistical significance testing is a widely used analysis to assess the obtained performance improvements. In the literature, the t-test is the most commonly used method for statistical significance [42], [46], [47]. The t-test quantifies the difference between the performance values of the two samples. On the other hand, the p-value quantifies the probability of occurrence as more extreme values assuming the null hypothesis is true. A p-value greater than a chosen threshold indicates that the observation is not so unlikely to have occurred by chance. Therefore, we do not reject the null hypothesis of equal population means. If the p-value is smaller than the threshold, we have evidence against the null hypothesis of equal population means. There are two types of t-tests, which are dependent and independent, according to the values that are obtained from the same or different environments. In this work, we evaluate all approaches with the same dataset, so we use the dependent t-test to assess the statistical significance of our results.
The significance test is applied as follows: The difference of significance in results is investigated by using f-measure results. Our null hypothesis is that there is no difference in our proposed methods and the other compared approaches. A significance level is computed by considering the f-measure value with respect to different recommendation counts in our experiments. The two-sided design is preferred to test whether the results are greater than or less than each other. The significance test is applied for each dataset separately. t-test score (t-value), and p-value are used to determine the significance. 1 When the significance level is low, the null hypothesis is rejected, and it is proved the proposed approach achieves statistically significant improvements.
We used the significance level α = .05 and found the critical value = 2.571 to determine the significance of SgWalk Results. t-test assessment results of SgWalk are given in Table 6

H. VALIDITY THREATS AND LIMITATIONS
SgWalk is a subgraph-based approach, and subgraphs are determined according to the check-ins clustered with respect to the spatial information. The size of the created clusters needs to be relatively small compared with the whole graph for SgWalk in order to capture the contextual relationships and execute them efficiently. Therefore, the selected clustering algorithm should generate the clusters from the dataset that allows small-sized user subgraph construction. We used the density-based algorithm DBSCAN to create clusters, and SgWalk has the best result with the four datasets used in experiments, with a minimum 7% improvement overall cases under @5 recommendations. On the other hand, if the method generates large clusters failing to fulfill small-sized subgraphs construction, possibly due to the density distribution of the dataset, SgWalk may not capture contextual relationships and perform as expected.
Since clustering is a crucial step in SgWalk, it constitutes a vulnerable point whose quality can affect the outcome. In the proposed setting, since the challenging problem is location recommendation for LBSN, the clustering is built on the spatial feature to extract the spatial context. For different settings and different contexts, other features of the social network can be used to decompose the full graph into smallsized subgraphs. For example, for a movie dataset, such subgraphs can be constructed by using the genre (with subgenres) feature if a variety of genres are provided within the dataset.

V. CONCLUSION
In this work, we propose a location recommendation method, SgWalk, which makes use of user subgraph based node embeddings. In the heart of the method lies a novel node embedding technique for HINs. The primary motivation in the proposed embedding technique is to be able to capture interactions among the heterogeneous type of nodes according to spatial context. Node embeddings are popularly VOLUME 9, 2021 used in HIN related problems, and the embedding methods in the literature generally utilize bi-partite subgraphs and meta-paths for random walk generation and use the entire graph to generate walk sequences. Such approaches rely on meta-path structure assumptions and do not focus on contextual relationships. To be able to incorporate such relationships into the embedding, SgWalk makes use of contextual user subgraphs. It does not require any meta-path assumption, as well. The use of subgraphs, which are smaller structures including fewer nodes and edges than the whole graph, is also helpful to improve computation time. The SgWalk technique follows four steps, which are (1) user subgraph construction (2) random walk sequences generation (3) learning graph embeddings (4) location recommendation generation, exploring the proximity between users and locations to provide recommendations.
In the proposed method, constructed user subgraph considers a wide range of information sources, including personal, social, spatial, and trustworthiness concepts. Such heterogeneous entities are frequently used in graph based recommendation methods. However, the use of trust notion in location recommendation studies is scarce in the literature. Therefore, the inclusion of trusted user nodes in the LBSN graph and using the node embedding based setting is another novelty of the study.
We evaluate our location recommendation method in comparison to the state-of-the-art methods on well-known datasets filtered for New York City. The experiments show that our proposed algorithm, SgWalk, provides the highest accuracy against compared methods for the majority of test cases, and the results indicate that our method brings a minimum 23% improvement for location recommendation accuracy in terms of f-measure@5. The analysis of the results reveals that the obtained improvement by SgWalk is statistically significant (α = .05) for 75% of the test cases. The experimental results show a strong indication that SgWalk, focusing on spatial context and generating node embeddings by using subgraphs with respect to spatial context, can capture the contextual relationships more effectively. Hence, it provides more accurate personalized and contextual location recommendations.
As future work, we plan to apply user-based subgraph generation in different domains such as movie and authorship databases for other recommendation tasks (such as movie recommendation to users and topic recommendation to authors). Additionally, studying the effect of temporal context nodes in the user subgraphs within the scope of LBSNs is another interesting research direction.