Decentralized Collaborative Filtering Algorithms Based on Complex Network Modeling and Degree Centrality

Given that everyone online is saturated with information, the theoretical significance of recommendation algorithms is evident in the fact that users need help finding products and content they care about. Collaborative filtering predicts a user’s rating on an item by finding similar users that rated the item or similar items that were rated by the user, and using the selected similar neighbors to “collaboratively filter” the recommendation. In the process, selected neighbors are considered equally important despite their differences in popularity. Here, we explore a method of modeling recommender systems as networks that can be constructed by considering items as nodes and similarity between them as links. Our research shows that item centrality has a negative impact on the accuracy of rating predictions, which needs to be considered for better algorithm performance. Experiments show that collaborative filtering algorithms can be decentralized by our method and provide a better accuracy of rating prediction. Furthermore, the relationship between the prediction target and its neighbors can be further evaluated based on both their similarity and their centrality.


I. INTRODUCTION
Recommender Systems (RSs) have received increasing attention in recent years. The systems have critical roles for most websites and e-commerce sites, such as online dating [1], movie recommendations [2]- [6], the evaluation of temporal networks [7], [8], collaborative recommendations [9] and so on. Technology titans like Amazon, Netflix and Taobao are using similar systems and algorithms to find new customers as well as selling more merchandise to old ones.
Consequently, recommendation algorithms have become one of the hot topics and have been studied in the past decade by scientists and engineers from different backgrounds. Various algorithms are being explored and presented by the community. Computer science researchers resolve the problem by using collaborative filtering and content-based filtering. The main ideas of those algorithms are making automatic predictions about users' preferences or items' potential customers.
The associate editor coordinating the review of this manuscript and approving it for publication was Chaitanya U. Kshirsagar.
The predictions are made based on the collected information of many other users or other items, respectively [10], [11].
In contrast, physicists attempt to solve the problem with users and items considered as nodes in networks, and to predict the connection and weight of the connection between a user and an item, also known as the link prediction [12], [13].
Recommendation algorithms can be classified into three types based on the main data sources used in predictions [11], [14]- [16].
• Content-based recommendation algorithms • Collaborative filtering recommendation algorithms • Hybrid recommendation algorithms Content-based (CB) recommendation algorithms focus on finding items with similar content or characteristics to items positively rated by target users [17], [18]. Data used in CB prediction is item content, such as the key words of papers, genres of movies and musics and so on. People's preference on items with such content is the foundation of possible recommendations.
Similarly, Collaborative Filtering (CF) methods treat the recommendation problem as making automatic predictions (also known as filtering) about the interests of the target user by collecting preferences or taste information from many other users (also known as collaborating) [19]. The ''collaborative'' refers to the collaboration effort of a group of users to ''filter'' the abundant data and make recommendations to the target user. Data used in CF methods is ratings or likes from users on different items. A group of similar users or items is the foundation to make the prediction.
Furthermore, the hybrid algorithms combine different content-based and CF methods to exploit information of both users and items [20], [21]. Various methods are used in this category to improve the recommendation the accuracy of the recommendations or improve the performance [22], [23]. Or targeting specific problems like cold-start problems [5], [24] or sparsity of data [15], [24], [25].
• Memory-based CF • Model-based CF The memory-based algorithms perform predictions based on the training data set, which is usually loaded into the memory [15], [23]. Similarity is usually calculated between each pair of users or items in this type of CF. And topk neighbors with the highest similarity are selected for the target user u [2]. However, some literature uses the k Nearest Neighbor (kNN) algorithm to find the most similar neighbors for the target user [10], [23].
The model-based CF algorithms aim to find patterns in the data and construct a model to predict new ratings [27]. Models like Matrix Factorization (MF) methods, slope one algorithms, complex network modeling, Bayesian probabilistic and neural networks are all used in different approaches and backgrounds.
However, complex network modeling and node centrality have not been used together to explore possible improvements in recommender systems in recent works.
Our main contributions are presented based on the previous analysis as follows: 1) We explore the centrality of nodes in complex network models to improve rating prediction in RS. 2) Our research on item-item network modeling in recommender systems indicates popular items is overrated in the prediction process and need a corresponding correction. 3) A framework is proposed to improve the accuracy of most memory-based CF methods.

A. COMPLEX NETWORK MODELING AND CENTRALITY
Networks of coupled dynamical systems have been used to model various complex systems [28], such as biological oscillators, excitable media, neural networks, software structures [29]- [31], spatial games, genetic control networks, collaboration networks in science [32], real-world airport network [33], the Internet topology [34] and even recommender systems [4], [35]. Studies [4] and [5] both use complex networks to model recommender systems as an item-item or a user-user network. Prediction of ratings in those algorithms takes network characteristics into consideration. For example, [5] constructed an item-item network and calculated the spatial distribution of item nodes, and used their distance in-between to replace similarity in predictions. [4] calculated the community structure [36]- [39] of item-item networks to adjust neighbor's weight in predictions. Both methods improve the accuracy of rating prediction, which means that the network modeling in recommender systems can reveal information to better understand of the systems.
Centrality metrics in network science provide methods to quantitatively measure node importance in the networks [36], [40]- [44]. One can use centrality measures to rank critical points in real-world complex systems [30], [45], or measure popularity of items by degree in recommender systems [11].
Since recommender systems can be considered as a complex networks [4], [5] with items or users as nodes and ratings or preferences as connections in-between, we can definitely design a model to construct an item-item network for further analysis of the system. With degree centrality considered as an indicator of item popularity, the relationship between node degree and prediction accuracy can be analyzed.

B. RELATED COLLABORATIVE FILTERING METHODS
He et al. in their work [2] proposed a method in 2015 based on similarity and top k similar neighbors, assuming opinion spread among users in recommender systems. The method provides more accurate predictions compared to some existing literature [2], [11]. This approach is considered as a memory-based CF, because the core idea is based on a similarity equation measuring the opinion spread between users. The top k neighbors are chosen for prediction instead of selection by kNN algorithms.
If an item-based CF is chosen, the algorithm follows a traditional pattern of memory-based CF, with an additional normalization of ratings to counteract user bias. After the normalization, a) similarity between users is calculated, b) the top k neighbors are selected for the target item i, c) for each user u not rated the target item i, calculate its prediction based on the ratings of k neighbors by user u.
where r max u and r min u stands for maximum and minimum rating value from user u such that e ui ∈ [−1, 1]. If r max u equals to r min u , e ui is assumed to be zero. VOLUME 8, 2020 Further, Equation 2 is used to capture the similarity between the items in [2].
a ui · a uj · e ui · e uj c u (2) where c i is the degree of item i and c u is the degree of user u in a bipartite graph consisting of users and items. a ui and a uj indicate whether or not user u has rated item i and item j, respectively. a ui = 1 if user u has rated i, otherwise a ui = 0. Therefore, the equation consider the similarity between user i and j is proportional to the sum of the products of the normalized scores of the jointly rated items. a ui and a uj restrict that both items must be rated by the same user. The two normalized ratings e ui and e uj must be sufficiently large enough and have the same signal to make a significant contribution to the similarity results. Moreover, Equation 2 is tuned slightly to be more precise as follows [3].

III. PROPOSED METHOD A. RECOMMENDER SYSTEM DESCRIPTION
Suppose there are t users and m items in a recommender system, the user set and item set can be denoted as U = Moreover, rating matrix R = {r ui } t×m ∈ R t×m is a t multiply m real matrix, where r ui is the element at the u − th row and i − th column, and r ui indicates the rating of user u on item i. The value of r ui ranges from 0.5 to 5.
Correspondingly, a matrix A = {a ui } t×m ∈ R t×m is a t multiply m binary matrix, and a ui stands for the element at the u − th row and the i − th column of the matrix. The value of this element can be defined as follows.
where u = 1, . . . , t and i = 1, . . . , m. It is important to mention that the data used in the similarity calculation can be either the ratings between two targets or content-related data like the genres of movie. The method is convenient for establishing links between items or users. However, the similarity exists between each pair of targets, and the item-item networks constructed by similarity results is a fully connected network as a result. The networks with n nodes have n·(n−1) 2 , n · (n − 1) links in undirected and directed networks, respectively. From the perspective of network science, there are too many links in the network to calculate meaningful centrality.
Thus, we need to control the number of links in networks to analyze proper network structure and centrality, by removing links with a weight less than zero or a threshold value. The structure of the topology is becoming more apparent with the smaller weight links gradually removed in Figure 1.
From the left to the right, the link weights in subfigures are assumed to be greater than 0, 0.5, 0.7, respectively. The similarity can be obtained between each pair of movies. We treat the similarities as links and their weights, and construct an item-item network. Due the similarity ranges between one and zero, the weights of the links are different and varied. This figure shows that the structure of topology is becoming clearer with smaller weight links removed gradually (from zero to any links with weights less than 0.7).
However, some nodes in the network have only small weight links, due to the factor that their style, content, or ratings are peculiar than others. Notably, when all links belonged to such kind of nodes are removed, experiments show that isolated nodes in the modeled network account for a large proportion of observed nodes.
In order to avoid this phenomenon and to keep the topology connected, two parameters q and p are introduced. q is the ratio of links to nodes which we want to keep in the network. p is the minimum number of links that each node must have in the network after link removing. Therefore, q and p can be used to determine how many links are in the network as well as maintaining the minimum number of links for each item.
The following four tasks are performed in the process of modeling item-item networks in RS. (1) determine similarity between each item pair in RS; (2) for each item i, keep the p links with the highest weight; (3) maintain the top n · q−n · p links in the network (typically the q is larger than p); and (4) remove all other links to make the topology clear.
According to the experimental results in [5], p ∈ {1, 2, 3} and q ∈ [1,20] are appropriate for typical recommender systems. p larger than three or q larger than twenty causes the network to become entangled with too many links, p = 0 causing the subgraphs to become disconnected from one another. The larger the q is, the larger the average degree of the network.
An modeled network example is shown in Figure 2, which confirms the effectiveness of our method for modeling the network from RS.

C. CONSIDER CENTRALITY AS A INDICATOR OF A NEIGHBOR's POPULARITY AND STATUS
The movie 'Toy Story' is a very large node at the bottom of Figure 2, and the high degree of the movie indicates that 'Toy Story' is rather popular and its status is important in the network. Further research is presented to explore whether such popularity and status affect the accuracy of rating predictions.
In our research, we use a simple centrality method, degree centrality, to test our assumption. The degree of items is normalized due to a possible wide range of node connection number.
For a graph G with n nodes, the normalized degree centrality for node n i is defined as follows.
Degree * (n i ) = M n i max n∈N (Degree(n)) + 1 (5) where M n i = Degree(n i ) is the degree of node n i representing the number of nodes connected with n i , and n is the number of nodes in the graph G. max(Degree(n)) is the maximum degree of nodes in the network. Thus, Degree * (n i ) is the normalized degree centrality that is assumed to be less than 1.0. Therefore, we propose a function to measure the centrality of item node as follows.
where Degree * (j) is the normalized degree centrality value of j. F is a set of functions, in which there are three functions can be used for possible prediction. For simplicity in experiments, we denote C a (j) = 1 Degree * (j)+1 , C b (j) = 1 − Degree * (j) and C c (j) = Degree * (j).
The reason that we design function C(j) is we are uncertain whether the popularity of some item will emphasize or de-emphasize the importance of an item.

D. DECENTRALIZATION OF COLLABORATIVE FILTERING METHODS
First, a prediction method is given by Equation 7 [2].
n j=1 s ji (7) where r i and r j are the average item ratings in the training data set, n is the number of selected neighbors of item i. Algorithms select top k neighbors of item i with the highest similarity s ji between item j and item i, which could be calculated by Equation 2 or Equation 3. Equation 8 [4] improved the prediction equation by identifying communities of nodes in the item-item network of RS.
n j=1 s ji (8) where ϕ is a function that increases or decreases the weight of similarity between item i and item j based on whether or not they are in the same community.
which is used to adjust the similarity between each pair of items. m i and m j donates item i's and item j's community numbers, respectively. If the degree centrality did affect the accuracy of rating prediction, we could use an algorithm to neutralize the possible adverse impact in the prediction.
As a result, the decentralized equations can be improved as follows to achieve the goal.
n j=1 s ji (9) or with communities considered, n j=1 s ji (10) where C(j) is a weight function. The value of C(j) is determined by the centrality of item j. We call the method as Decentralized Collaborative Filtering (DCF) which corresponds to the classical Collaborative Filtering (CF) algorithms.
Theoretically, the method can be applied to any Collaborative Filtering (CF) with similarity calculations. In a corresponding DCF to the original CF method, the following four tasks are executed: (1) determine similarity between each pair of items or users in RS; (2) model an item-item or user-user network based on similarities and remove unnecessary links; (3) calculate the degree of nodes in the network; and (4) predict ratings based on Equation 9 or similar equations. VOLUME 8, 2020

A. DATASET
To test the performance of the presented algorithm, both MovieLens and Netflix are chosen as benchmark data sets.
The MovieLens had 671 users and 9125 movies, and 10 5 ratings distributed from 0.5 to 5. The Netflix data set consisted of 124600 ratings, 1246 users and 3100 movies, which is a random sample from the original Netflix data set.
To test presented algorithm appropriately, a 10-fold cross-validation is applied to the data set, and the training set is randomly divided into 10 clusters. Each data cluster is used for testing and the remaining nine clusters are used for training. Any algorithm in the test predicts each cluster based on the information provided by the other 9 clusters, in which the algorithm picks the top k neighbors (k ∈

B. COMPARISON ALGORITHMS
To validate and test our methods and compare with some state-of-the-art algorithms, CF-Pearson, CF-Cosine, CF-OS, CF-OS and CF-MIOS are used as the abbreviations to represent several recent or standard methods using Pearson correlation collaborative filtering, Cosine correlation collaborative filtering, Opinion Spreading [2], and Modularized Improved Opinion Spreading [4], respectively. The latter two algorithms were proposed in 2015, and 2019.
Because the proposed method balanced the items or users with its own degree centrality, it is named as decentralized collaborative filtering algorithms based on complex network modeling and degree centrality, which is abbreviated as DCF.  [51] are also used to compare the optimal accuracy of prediction, which provide a broad picture of the algorithms' performance.
We used a machine with Intel(R) Core (TM) i9-9900K CPU @ 4.8GHz with 32GB main memory. The algorithms are implemented in FSharp language running on a Windows 10 x64 operating system. All the parameters affecting the computation time were set the same for all algorithms whenever possible.

C. PERFORMANCE EVALUATION
Two metrics are chosen to measure the prediction accuracy, namely, Normalized Mean Absolute Errors (NMAE) and Normalized Root Mean Squared Errors (NRMSE). Using NMAE and NRMSE instead of MAE and RMSE, we can comfortably compare the optimal accuracy of the proposed method with some existing work without unnecessary programming [11].
By comparing the actual ratings given by users to the predicted ratings given by algorithms, the smaller the NMAE and NRMSE are, the more accurate the algorithms are. NRMSE squares the prediction error before the summation and thus makes the large errors more obvious.
where T set is the test set, r ui is the predicted rating andr ui is the real rating in the test data set. R max and R min are the maximum rating and the minimum rating in the system, respectively.

D. PERFORMANCE OF DIFFERENT DECENTRALIZED FUNCTIONS
The prediction results shown in Fig.3 and Fig.4 are all itembased, and the results of NMAE and NRMSE show the error level of the target methods on the MovieLens dataset. DCF-MIOS with C a (j) and DCF-MIOS with C b (j) are far superior than the original MIOS, suggesting popularity of the items is overstated in their importance. As such, their status should be de-emphasized in the rating prediction to improve accuracy. Approaches with C a (i), C b (i) are preferable fixes for decentralizing node popularity in RS. According to our experimental results, C a (i) has the best performance among the three functions. Therefore, we use C a (i) in the following sections without further indication.

E. DECENTRALIZED COLLABORATIVE FILTERING IMPROVES THE ACCURACY OF THE CORRESPONDING METHODS
In Fig. 5 and Fig. 6, we introduce more algorithms into testing. Results from the NMAE and NRMSE indicate the accuracy of the methodology in the MovieLens data set. All   four methods are improved by using network modeling and node decentralization.
In this section, the parameters p = 1 and q = 1.5 are used. They are determined by a group of experiments. For the sake of clarity, the experiments are omitted in this article.
This experiment confirms that our method can be used in different memory-based CF methods to improve the prediction accuracy.  but the improvements of DCF-IOS are relatively small. We concluded after analysis that this was due to the lack of a clear community structure in the sample. Nevertheless, the optimal accuracy of DCF-IOS is still a little better than that of CF-MIOS.

G. THE OPTIMAL ACCURACY COMPARISON
The most important results from the experiments are the optimal ones with the smallest errors in rating prediction. Thus, Table 1 shows the result comparison of the optimal accuracy with the top k neighbors among most of the aforementioned algorithms (k may vary by different methods).  Table 1 and Table 2 confirm the effectiveness of the proposed methods that can be used to improve the various memory-based CF approaches. DCF-MIOS has the best NMAE, and NRMSE has improved 14% compared to the original CF-MIOS, which is only 4% percent worse than BPMF in model-based category. The number of neighbors used in the experiments in the table is different.

V. CONCLUSION
Traditional CF methods only consider them based on similarities and cause accuracy losses. Our improvement in VOLUME 8, 2020 prediction accuracy is based on the fact that items in the recommender system differ in popularity, which can be revealed by their centrality in network models.
Furthermore, the additional time complexity of our methods mainly comes from removing irrelevant links and calculating the degree of nodes in the modeled network. The time complexity of sorting link importance and removing links is O(n · log(n)), and the time complexity of calculating node degree is O(n) in the worst case scenario, which is acceptable even in large systems.
To compare various nodes differently according to their centrality in the networks, we designed a model to predict user ratings for items, which is capable of improving the predictive accuracy of various memory-based CF methods. The algorithm uses the information from the ratings to calculate the similarity of the targets, and constructs a network based on the similarity results. As a result, centrality metrics can be used in the prediction process, which intriguingly increases the prediction accuracy for every algorithm tested in our experiments.
In particular, the research demonstrates that the inequality of items is nontrivial in recommender systems. Popular item nodes are overrated in prediction without considering their centrality.
Theoretically, our model can be applied to many recommendation algorithms. With similarity results as link weights and the modeling of complex networks, centrality metrics can be taken into consideration to provide better prediction accuracy.