A Recommendation Strategy Integrating Higher-Order Feature Interactions With Knowledge Graphs

Knowledge Graphs (KG) are efficient auxiliary information in recommender systems. However, in knowledge graph feature learning, a major objective is improvement for recommendation performance. One of the problems in the existing methods is that they cannot uncover the deep interaction information of users in a simple way, and this motivates effective learning of the potential embedded information through the knowledge graph. The Graph Convolutional Network (GCN) can be useful for learning information about graph structured data. This paper proposes a method that fuses higher order feature interactions and knowledge graphs and uses them for recommendation. For users, they uses Gated Recurrent Units (GRU) to focus on their preferences so that the ability of convolutional neural networks in processing user preference features is enhanced; for items, the cross-learning module is adopted to learn higher order features between items and entities; for users and entities, KG and user-item interaction information are combined followed by feature extraction of graph structured data by Light GCN, allowing the model to learn potential user-entity associations from the graph structured data. Current experiments on two real datasets show that the proposed model performs better than some recently developed models.


I. INTRODUCTION
Online information browsing has permeated people's daily lives as the Internet era has progressed [1]. The information overload is becoming an issue due to the growth of the Internet, making it difficult for users to sift through the mass of information to find their truly interests. Recommendation systems have been developed as a result of the urgent requirement for an effective information filtering mechanism [2]. The aim of recommendation system is to extract the user's The associate editor coordinating the review of this manuscript and approving it for publication was Alireza Sadeghian. needs and interests from their previous interactions with the items and then provide the findings to the user. Contentbased recommendation [24], [25], [26], [37], collaborative filtering-based recommendation [24], [38], and their combinations [27], [28], [40], [41] are three primary types of recommendation algorithms. The recommendation accuracy keeps increasing from the initial periods of machine learning to the present day of deep learning [8]. Aiming to address the sufferings from data sparsity [43] and cold-start issues, a common solution is to incorporate different auxiliary information [40] into recommendation models, including user/item attributes, social networks or contextual information.
Knowledge graphs could offer logical justifications for the recommendation outcomes since they have structured semantic knowledge of auxiliary information.For the cold start problem, most of the existing solutions are achieved by enhancing knowledge embedding, adopting heterogeneous graphs to enrich data representation or using Graph Neural Networks (GCN). In this paper, we focus on alleviating the cold start problem by introducing graph convolutional neural networks to explore users' potential interests.
Major types of knowledge graph-based recommendation systems include the embedding-based method and the path based approach. In embedding-based techniques, Knowledge graph embedding is carried out to preprocess KG, and the item embedding representation is subsequently used in recommendations. In order to obtain an embedding representation, Collaborative Knowledge Base Embedding (CKE) [6] analyzes the structural data of KG using TransR [3], while Deep Knowledge-aware Network (DKN) [11] builds a knowledge graph by extracting text data and fusing the word vectors of news with the entities of the knowledge graph. The path-based approach involves constructing a user-item graph and using the connectivity of entities in the graph to make recommendations. The recommendation results are improved by the similarity of connections between users and/or items, but this approach heavily depends on manually created meta-paths. The ''matrix factorization (MF) + factorization machine (FM)'' method is used by Meta-Graph Based Recommendation Fusion [12] to integrate meta-graph into the Heterogeneous Information Network (HIN) based recommendation system and address the information fusion issues. The MF technique is used to produce potential features of users and possible elements of items for each meta-graph similarity. A group lasso regularization-based FM approach is presented to automatically choose the appropriate metagraph method from the observed data for various metagraph attributes. When coding a given entity representation, Wang et al. [29] proposes a model Knowledge Graph Convolutional Networks (KGCN) that fuses KG features and graph convolutional neural networks to integrate neighbour information and deviations to acquire neighbourhood structure more effectively. This model enables users to receive more personalized interest recommendations.
In recent years, deep learning has been increasingly applied to recommendation algorithms owing to the widespread use of these techniques in image processing, machine translation, and intelligent questioning and answering systems. The enhanced efficiency of recommendations originates from neural networks' effective access to deep semantic information among data. Yang et al. [14] introduces a novel Knowledge-enhanced Deep recommendation framework incorporating Generative and Adversarial Network (KTGAN), which is a GAN-based model with two stages. The first stage is to learn KG embedding in combination with Metapath2Vec [36] and tag embedding in combination with Word2Vec [35], while the second stage uses a generator G and a discriminator D to refine the user's initial representation. The knowledge-enhanced Sequential Recommender (KSR) [15] framework combines Gated Recurrent Units (GRU) network with a Key-Value memory network, where the GRU network is used to capture the serialized preferences of the user, and the Key-Value memory network models the attribute-level preferences of the user through Trans E [34]. In this work, we introduce knowledge graphs as auxiliary information into the recommendation model and apply deep learning techniques to learn user-item interaction information from the supplementary data. The main contributions of this paper are as follow: (1) The cross-learning module is proposed for mining explicit and implicit higher-order interactions between items, which provides auxiliary information for recommendations.
(2) To enrich the relationship between users and entities, a user-item-entity graph is constructed, and a graph feature learning module is used to explore the semantic relationships in the graph structure data to better improve the recommendation performance.
(3) We conduct extensive experiments on two benchmark datasets, and the results show that the proposed model outperforms the other benchmark models.

II. RELATED WORK A. RECOMMENDER SYSTEM
With the development and application of deep learning technology, neural networks have been applied to all aspects of recommendation systems, and the recommendation algorithms have experienced from content-based recommendations to collaborative filtering-based recommendations, which are further developed from user-and item-based collaborative filtering to model-based Collaborative Filtering (CF). Since deep learning can effectively mine data features, researchers have introduced large amount of auxiliary information, such as text, tags, social relationships, etc., to extract favorable features for recommendations. ConvMF [18] model embeds convolutional neural networks into maximum likelihood estimation to generate recommendations based on review text analysis. Zhou et al. proposes Deep Interest Network (DIN) [19], presenting a deep learning method based on mini-batch regularization and data-adaptive activation function, which adds an attention mechanism after the embedding layer, aiming to generate user features from the recommended items. The Wide&Deep [20] model merged linear models and Deep Neural Network (DNN) models, which utilize the memory capability of the shallow model and the generalization capability of the deep model in a comprehensive way, enabling the proposed model to combine the features of accuracy and scalability, and thus achieving improved feature extraction capabilities. Knowledge-aware Path Recurrent Network (KPRN) [21], which uses knowledge graphs as auxiliary information for recommendation, employs Recurrent Neural Networks (RNN) to learn users' preferences, and finally uses path information to enhance the interpretability of recommendations. MKR [22] is a multi-task learning recommendation framework that combines the knowledge graph learning task with the recommendation task well for training, resulting in improved performance of the final recommendation.

B. KNOWLEDGE GRAPH-BASED RECOMMENDATION
Knowledge graph has powerful semantic expression capability and is essentially a knowledge base of semantic network. Google proposed the knowledge graph concept in 2012, aiming to use the knowledge base to improve the performance of search engines and enhance the user's search efficiency [44]. With the development and application of artificial intelligence, knowledge graph has been widely used in intelligent search, intelligent Q&A and personalized recommendation [44]. Presently, the application of knowledge graph feature learning to recommender systems is mainly through three methods, i.e. sequential learning, joint learning, and multi-task learning, with typical DKN, CKE, and MKR models, respectively. In this study, we will conduct modeling by using a multi-task learning strategy.
The embedding-based approaches [6], [7], [9], [10], [11], [13], [15] characterize entities and relationships by knowledge graph embedding (KGE) [3], [4], [34], which in turn obtains additional semantic information. Relations provide additional information for knowledge graph-based recommendations that can use inter-node reasoning to discover new connections [45]. DKFM [7] is proposed for points of interest (POI) recommendation, using TransE [34] to train urban data for enriching the representation of destinations as well as for improving the relevant recommendation performance. BEM [10] uses two types of graph data, i.e. knowledgerelated graphs and behavioral graphs, for firstly learning the initial embedding representation from the graph using TransE and Graph Neural Networks (GNN), respectively. After that, a Bayesian framework is used to extract the two types of embeddings. KTUP [9] uses TransH [4] to jointly learn the enhanced item-preference embeddings and the entity-relationship embeddings. Path-based approaches [5], [8], [17], [18] aim to uncover multiple connections between users and items based on graphs to explore potential information in knowledge graphs. McRec [5] designs a mutual attention mechanism based on meta-paths for top-K recommendations, extracts potential features of meta-paths, and exploits connectivity between users and different types of relational paths or items on graphs. The KPRN [17] exploits the sequential dependencies in paths, making it possible to reason effectively over the paths. Although the path-based approach makes more natural use of the knowledge graph, but it relies heavily on the design of meta-paths, which require specialized domain knowledge for effective meta-paths.

III. PROBLEM FORMULATION
In the study of knowledge graph-enhanced recommendations for a particular recommendation scenario, we have a collection of users U = {u 1 , u 2 , . . . , u m } and a group of items V = {v 1 , v 2 , . . . , v n }; where m and n denote the corresponding numbers. Typically, the user-item interaction data is represented as a bipartite graph, and the user-item interaction matrix Y ∈ R m×n is established based on implicit user feedback.
The presence of the y ui = 1 here denotes user interaction with the item, which may take actions like browsing, clicking, purchasing, rating, etc; and y ui = 0 indicates that the user did not engage with the item, yet it does not imply that the user disapproved of the item. It is also possible that the user did not navigate to the item since no interaction took place. The knowledge graph G is made up of entity-relationshipentity triples (h, r, t), with h, r, and t standing for the head entity, relationship, and tail entity, respectively. For example, the triplet (Paul Kalanithi, book. written_work. author, When Breath Becomes Air) identifies Paul Kalanithi as the writer of ''When Breath Becomes Air''. Knowledge graphs provide factual knowledge and rich semantic information about items. In many recommendation scenarios, an item v ∈ V corresponding to one or more entities, so that we could establish item-entity alignments. Through the alignment of items and knowledge graph entities, the knowledge graph can provide complementary information for interaction information.
The challenge of our knowledge-aware recommendation is to forecast the potential interest of a user u in an unobserved item v given a user-item interaction matrix Y and a knowledge graph G. The aim is to train a prediction functionŷ uv = where θ is the model parameter of the function F, to calculate the chance of user u selecting item v. Table 1 lists a description of each symbol used in this paper.

IV. METHODOLOGY
In this section, we provide an overview of the general framework before detailing our proposed approach and presenting the implementation details of the specific modules individually.

A. AN OVERVIEW
The general framework of our model is shown in Figure 1, which consists of three main tasks; that is, knowledge graph embedding learning, feature learning of graph data, and recommendation module. (a) Knowledge graph embedding learning is a task that mainly focuses on learning the knowledge graph tripple data and linking the knowledge graph feature learning with the recommendation task through a Cross Item-Feature Learning (CIFL) module for data sharing and parameter transfer to achieve better recommendations. For this task, the head h and relation r of the knowledge graph are used as input, and h and r are obtained through processing the CIFL module and Convolutional Neural Networks (CNN) respectively. A Multilayer Perceptron (MLP) is implemented to predict the tail, with the use of a similarity function as the evaluation of the final result. (b) Graph data feature learning module, through combining the knowledge graph data and user-item interaction data into a user-item-entity graph, to process the user-item-entity graph by a Light GCN network for extracting the fine-grained information of users and items. (c) Recommendation module is linked with the knowledge graph embedding learning module through the CIFL module. The user and item embedding vectors obtained through the graph data feature learning module are stitched together with the current user and item embedding vectors, and finally the final results are predicted using a multilayer perceptron.

B. GRAPH DATA FEATURE LEARNING
The items might be represented by how the users interact; however, in the past, only the user-item graph was taken into account. The representation of how information is interacted with is too broad, so we wish to know the fine-grained characteristics more thoroughly. In order to build the user-itementity graph, which preserves more comprehensive structural information, we integrate the user-item interaction graph with the knowledge graph. Therefore, in order to automatically encode the route information within the nodes and maintain the information of nearby nodes and the user-item-entity path information, we would treat the structural information using Light GCN, a graph encoder with path-awareness. In general, graph data feature learning is to learn the relationship between users and entities in the graph, so as to explore the deeper features of users. The light GCN is a light yet efficient GCN network that forgoes nonlinear activation and typical feature transformation in favor of a simple message transmission mechanism and the most fundamental GCN aggregation formulation at the k layer, is illustrated in Equation 2.
where N u and N v represent the set of items that user u and item v have interacted with, respectively, and e (k) v and e (k) u denote the embeddings of user u and item v at the k th level. The symmetric regularization term follows the standard GCN structure with the purpose of avoiding the size of the embedding to increase with the graph convolution operation. We then sum over them to obtain the most embedding representations of the users and the items.

C. CROSS ITEM-FEATURE LEARNING
Based on the user-item interaction information, we can infer the user's item preferences. To better learn the interaction information between items and entities, we design a CIFL module, which models the higher-order interactions between items and their corresponding entity features. We utilize the main module of XDeepFM [23], a Compressed Interaction Network (CIN), for item preference extraction with explicit higher-order features. The idea of CIN is to achieve automatic explicative learning of higher-order feature interactions so that the feature interactions occur between individual feature vectors, i.e., feature interactions at the vector level. The CIN network includes CNN and RNN, which can adjust the feature order by controlling the number of layers of the neural network so that feature interactions occur vector-wise, effectively avoiding the defect that the number of model parameters increases exponentially with the number of layers, thus achieving the purpose of efficient extraction. In the CIN module, the output of each layer is used as the input of the next layer, and the neurons of each layer are calculated based on the hidden layer of the previous layer and the original feature vector is calculated as shown below.
The original features of the input and hidden layers of the neural network are converted into matrices denoted as X 0 and X k , respectively, where the i-layer embedded feature vector is presented by X 0 i, * = e i , H k is the i-row of X 0 , H k is the number of feature vectors in the k layer, x k−1 i, * is the i layer row of the output matrix of the k − 1 layer in the CIN, X 0 j, * is the j-layer row feature vector of the original feature, and w k,h ∈ R H k−1 * m denotes the parameter matrix of the h th feature vector. Equation 6 allows us to divide the CIN model computation into three stages. The output state X k of the preceding hidden layer and the output matrix X 0 of the embedding layer is used to compute X k−1 i, * • X 0 j, * so as to provide an intermediate result of Z k+1 , which is the first stage. The output of the next hidden layer is produced in the second stage by performing a layer-by-layer feature mapping operation on the intermediate result of the first stage and using a convolution kernel of mH vectors at H k+1 positions. Each feature mapping matrix created in the second stage is added together and merged in the third stage to produce P k i = D j=1 X k i,j . A DNN is used to facilitate feature interactions created in the bit-wise level, allowing each feature to interact implicitly with other features and thereby satisfying the higher-order feature interactions between various components. This allows collecting of more data regarding the implicit interaction of items. The following equations are used to calculated implicit interaction information.
where W k stands for the weight between the k-layer and (k +1)-layer, and b k represents for the bias of the (k +1)-layer.
x k is the output of the k-layer, σ is the activation function. That is, the higher-order features are implicit higher-order interactions at the element level.

D. KNOWLEDGE GRAPH EMBEDDING MODULE
The knowledge graph, as a powerful knowledge network structure, mainly consists of the triad {(h, r, t) | h, t ∈ E, r ∈ R}, where E is the set of entities in the knowledge graph and R is the set of relations. The knowledge graph representation learning refers to embedding the entities and relations composing knowledge into a low-dimensional vector space while preserving the structural and semantic information of KG. In this work, a semantic matching-based knowledge graph learning model is used to mine the attribute features of items so as to find out the potential connections among items, connecting the knowledge graph feature learning module and the recommendation module through the CIFL module, and then obtaining the vector representation of items using the training method of multi-task learning. Firstly, the head h in the knowledge graph triple is processed using the CIFL module; secondly, the relationship r is processed using the convolution module; thirdly, h and r are stitched together; its dimensionality is reduced by an l-layer MLP, to obtain a prediction (t). The similarity function is used to evaluate the prediction results.
where C stands for the convolutional neural network, and F for the CIFL module. Equation 12 illustrates how a similarity function may be used to get the highest similarity of the triplet (h, r, t). σ is the sigmoid activation function and t is the prediction vector for tail t.

E. RECOMMENDATION MODULE
The inputs for the model are the original feature vectors of users and items, which are fed into the embedding layer of users and items to obtain the low-dimensional dense vector representations of users and items. The user embedding set is U ∈ R and the item embedding set is V ∈ R. The embedding vectors of users and items are represented as u ∈ R, v ∈ R. For different users, the items in the current sequence have different attention levels. Therefore, we use the gated cyclic unit based on the attention mechanism to obtain the item preferences for the current user in the sequence. We use CNN to obtain its preference feature distribution for the refined current preferences. For a given user's original feature vector u, the u gru and u cnn after GRU and CNN processing are as follows: u cnn = CNN u gru (14) For item v, we processed it using the CIFL module to obtain: The e u and e v obtained by the graph data feature learning process are stitched together with the u cnn and v l received by the recommendation module to obtain the final u and v. Finally, the u and v are stitched together by an l-layer MLP for user u click-through rate prediction.

F. PARAMETER LEARNING
The loss function of our model is as follows: where L RS is a measure of the loss value of the recommended model, L KG is the loss in the computed knowledge embedding, with J being the cross-entropy loss function, while the last term is a regularization term to prevent overfitting, with λ being the regularization term.
The learning parameters of this model are composed of two parts, i.e., the recommendation module and the KG module, and the training algorithm of our model is given in Algorithm 1. The parameters are shared and influence each other between these two parts. In our experiments, the model parameters are trained in a joint learning manner, i.e., joint training. The joint training optimizes the parameters of both the KG embedding part and the recommendation part during the training process, in such a way that the two parts can interact with each other to learn a more reliable representation of the final prediction. We also use alternate calls to the optimizer for multi-task learning, where in each iteration we first learn the features in the user-item-entity graph, then learn the knowledge embedding, and finally connect the two parts by a cross-learning module that performs gradient descent for parameter updating.

V. EXPERIMENTS
In this work, several experiments are carried out to assess the efficacy of our proposed models. In this section, we present the experimental data collection, assessment metrics, baseline approach, and implementation details. The experiments and results are then analyzed.

A. EXPERIMENT SETTINGS
For the purpose of evaluating the recommendation performance of our proposed model, we performed experiments on two benchmark datasets to answer the following questions. while i+batchSize < len(Y) do 6: Pass the miniBatch from Y into the recommendation module 7: Using the Adam optimizer, update the F parameter according to Eq.(2)-(5), (9)-(12) 8: i+ = batchSize 9: while i+batchSize < len(G) do 10: Pass the miniBatch from G into the recommendation module 11: Using the SGD optimizer, update the F parameter according to Eq.

1) DATASETS DESCRIPTION
We evaluate this model using two benchmark datasets, Book-Crossing and Last.FM. These two datasets are from open source and differ in size and sparsity, making our experimental results more convincing using this dataset from two different domains.
Book-Crossing 1 : Book-Crossing is a dataset containing more than 1 million displayed ratings of books, ranging from 1 to 10.
Last.FM 2 : This collection of user preferences data is from the online music service Last. FM, with each user's top artists and play counts included in the dataset.
Because of the interactions in Book-Crossing and Last.FM are explicit feedback, we transform them into implicit feedback according to RippleNet [8]; where 1 denotes a positive sample, and we do not set a positive scoring threshold for Book-Crossing and Last.FM data due to their sparsity. For the construction of sub-KG, we used Microsoft Satori where each sub-KG follows a triple. Each sub-KG follows a triple format and is a subset of the whole KG with a confidence level greater than 0.9. For a given sub-KG, we filter the triple with a good match from the sub-KG by comparing the name of the dataset with the head entity of the triple. The basic statistics of these two datasets are shown in Table 2.

2) EVALUATION METRICS
We assess our method in various experimental scenarios: (1) The threshold for a favorable rating in click-throughrate (CTR) prediction is set at 0.5. We would recommend item v to user u if y uv > = 0.5. We choose AUC and ACC (area under the ROC curve) as methods to evaluate CTR as we apply the training model to each interaction in the test sets. (2) In top-K recommendations, we pick Recall@K and Precision@K to assess the recommendation list and apply the trained model to choose the k items with the highest predicted click-through rate for each user in the test set. Equation 17 shows Recall is evaluated by the proportion of appropriate items recommended for the target user to the total number of relevant items. According to Equation 18, precision is the proportion of accurate item predictions to all item guesses.  [6] is a typical embedding-based recommendation method that uses structured, textual and visual contents to make recommendations.
NFM [30] is a DNN structure introduced on the basis of FM model to learn more data information using nonlinear structure. NFM uses bi-linear interaction structure for second-order cross information, which makes DNN structure able to learn cross feature information better and reduces the difficulty of DNN to learn higher-order cross feature information.
BPR [31] is a Bayesian personalized ranking matrix decomposition framework which proposes a general optimization criterion for personalized ranking, BPR-Opt. In order to maximize BPR-Opt, a general learning algorithm based on stochastic gradient descent, Learn BPR, is utilized.
KGAT [32] is a knowledge graph-based model which applies attention mechanism to knowledge graph convolution to construct higher-order relationships. ECFKG [33] constructs a user-item knowledge graph. In this KG, users, items and their related attributes are treated as entities, and users' historical behaviors such as purchase and mention are regarded as a type of relationship between entities.
MKR [22] is a recommendation algorithm based on multitask learning, which utilizes the recommendation module and the knowledge graph characterization module for alternate learning as a way to combine the user-item matching and knowledge graph embedding tasks.
CKAN [39] is built on the basis of KGNN-LS with independent collaboration propagation and knowledge signals on CF and KG, respectively, using a heterogeneous propagation strategy to encode both types of information and applying a knowledge-aware attention mechanism to distinguish the contributions of different knowledge neighbors.
KGIN [42] is to reveal user-item interactions at the intent granularity by aiding item knowledge, which models each intent as an attentional combination of KG relationships, enhances different intent independence, considering more fine-grained user-item relationships and remote semantics of relationship paths.

4) IMPLEMENTATION DETAILS
We train all our models on an NVIDIA GeForce RTX 2080 Ti with 11Gb of video memory; and for each dataset we divide it into training set, validation set, and testing set in a 6:2:2 ratio. We repeat each experiment 10 times and report the average results for tuning the key parameters. For comparisons, we have fixed the batch size of data for all of models as 128, initialized the embedding parameters using the Xavier method, and optimized our approach using the Adam optimizer. The optimal parameter settings were determined by grid search, adjusting the learning rate in 10 −6 , 10 −5 , 10 −4 , 10 −3 , 10 −2 and the aggregation depth of the user-item-entity graph in {1,2,3,4}. The optimal hyperparameter settings for the comparison experiments are either from the actual study or follow the original paper.

1) PERFORMANCE COMPARISON (RQ1)
We show the results of our proposed model and some state-of-the-art methods for CTR prediction experiments and   Table 3, Figure 2 and Figure 3. From the table, we can see that our proposed model outperforms all baselines on top-K and shows excellent performance on the CTR task. Knowledge-aware based models can obtain better performance, and we can observe from the experimental results that most of the knowledge graph based models outperform the traditional models, which indicates the introduction of knowledge graph can effectively enrich the relational information and effectively improve the recommendation. However, the simple integration of KG does not ensure the performance improvement, and the CF-based BPRMF approach is better than the embeddingbased CKE approach, which indicates that the imbalance of the knowledge graph may lead to the degradation of the model performance. From the experimental results, we can demonstrate that our approach obtains better performance than the method only based on collaborative knowledge graphs and proves the effectiveness of extracting higherorder interaction feature information using deep learning. The hybrid methods KGAT and MKR based on knowledge graphs obtain better performance compared to other methods. MKR outperforms the matrix decomposition-based model, showing the power of neural network models in recommendation systems. Compared with KGAT, our model makes a new attempt to capture fine-grained information in graph data using GCN and to obtain the higher-order user-item interaction information using DNN. While KGNN-LS shows better performance, indicating the power of GNN in graph data processing, further suggesting that richer KG factual information can improve model performance. For Last.FM and Book-Crossing, we consider positive implicit feedback when the user's score for the item is greater than 0.5.

TABLE 4.
Comparison of the recommended performance of the ablation models for the two datasets. The two methods lack graph structure data and DNN structure, respectively.

2) ABLATION ANALYSIS (RQ2)
To investigate whether both the graph feature learning module and the cross-learning module proposed in this method contribute to the performance of the model, we propose two simplified methods for comparison and set up the following comparison scenarios: 1) not using the user-item-entity graph structure in the recommendation module and using only the interaction graph of user-item information. 2) using only CIN for item preference extraction in the cross-learning module, and not using DNN. As can be seen from the table 4, removing the graph feature learning and cross-learning modules degrades the performance of the model. Specifically, in the top-K task, using the graph feature learning module outperforms the case when it is not used, indicating that a higherorder connected graph structure is effective in improving model performance and providing more feature descriptions. In the case without DNN, the model performance decreases compared to when it is used, as seen in the figure and table, indicating the importance of implicit higher-order features for inter-item connectivity.

3) HYPERPARAMETER SENSITIVITY (RQ3)
Embedding Size: in the neural network recommendation model, the embedding dimension as an important hyperparameter affects the model performance. Therefore, we verify the impact of different embedding dimensions on the performance of this model while ensuring that other parameters consistent with those mentioned in the previous section. The effects of varying embedding dimensions on the models AUC and ACC under the Book-Crossing dataset are shown in Figure 4. It can be seen from the figure that the performance of the model improves with the increase of embedding dimensions, and the best embedding dimension is 64. However, the performance of the model start to decrease at 64, which might be attributed to the overfitting and other problems brought by the increase of embedding dimensions, resulting in the VOLUME 10, 2022  decrease of the generalization ability of the model. Therefore, we choose 64 as the embedding dimension of the model. Aggregation depth: to investigate the effect of different aggregation depths on the user-item-entity graph, we studied the impact of depths on the model performance on the Book-Crossing and Last.FM datasets, taking values in the range {1, 2, 3, 4}. The empirical results show that the model works best on both datasets at L = 1, indicating that the aggregated information of the user-item-entity graph is expressed by first-order neighbors.
Learning rate: the learning rate as an important parameter affects the recommended performance, shown in Figure 6. From the figure, it is shown that the AUC and ACC performance of the model improves when the learning rate increases from 10 −4 to 10 −3 , and the performance decreases when it increases further. Therefore, a setting of 10 −3 is considered optimal.

VI. CONCLUSION AND FUTURE WORK
In this work, we propose a novel recommendation framework that incorporates higher-order feature interactions and knowledge graph feature learning, which jointly acts on the knowledge graph structure and user-item interaction information in a unified neural network model to achieve recommendations. Specifically, this paper proposes a knowledge graph feature learning module based on a combination of convolutional networks and cross-learning, which exploits the higher-order structural similarity between entities and items in the knowledge graph to enhance the embedding learning of items. We design a cross-learning module that uses CIN and DNN to learn both explicit and implicit higherorder feature interactions, which is used to link the knowledge graph feature learning and recommendation modules to enhance the recommendation performance. The parameters of the framework are all optimized by means of union, and the experimental results show that the framework can capture rich semantic information as well as the complex hidden relationships between users and recommended items well. The graph convolution method is also used to mine the higherorder connectivity of graph data to this recommendation framework. Finally, by conducting extensive experiments on two real datasets, the results demonstrate the effectiveness of our proposed model. In future work, we will work on top-K prediction, and deal with sparse data more effectively, using knowledge graph data, heterogeneous graphs and graph convolution to solve the cold start problem, and extending the framework proposed in this paper for article recommendation.
YANRU LIN received the bachelor's degree in network engineering from the Shangluo College, in 2020. She is currently pursuing the master's degree in computer technology with Ningbo University. Her current research interests include knowledge graph and recommendation systems. He is currently an Associate Researcher. He is also mainly engaging in the application of material informatics, intelligent optimization algorithms, and system design methods in the research and development of new ternary layered materials and extreme environmental energy materials. At present, he has published more than 40 papers in Acta Materialia, Materialia, Philosophical Magazine, Journal of Power Sources, and other magazines, with more than 400 citations, six patents, and five software copyrights. He is a member of the British Society of Minerals, Mining and Materials (IOM3).
KAI DUAN received the bachelor's degree in computer science and technology from Zhejiang Ocean University, in 2020. He is currently pursuing the master's degree in computer technology with Ningbo University. He is currently working on knowledge question answering systems and knowledge graph.
QING HUANG received the bachelor's degree in inorganic non-metallics and the master's degree in materials science from Tianjin University, in 1995 and 2002, respectively, and the Ph.D. degree in materials science from the Shanghai Institute of Ceramics, Chinese Academy of Sciences (CAS). From 2005 to 2010, he carried out postdoctoral research at the National Institute for Materials Science, Japan, and the University of California Davis, USA. In 2010, he joined the Ningbo Institute of Materials Technology and Engineering, and focused on the developments and applications of energy materials under extreme environments including the creation of non-van der waals layered materials (MAX phases and MXenes) and silicon carbide composites for high-safety energy systems. He is currently a Professor with the Ningbo Institute of Materials Technology and Engineering, CAS. He has published more than 20 papers in Information Processing and Management, Complexity, and other magazines, with more than 180 citations, and eight patents and two software copyrights. His current research interests include pattern recognition and machine learning and wireless sensor networks.