Memory Reorganization: A Symmetric Memory Network for Reorganizing Neighbors and Topics to Complete Rating Prediction

Using pre-trained topic information to assist in training neural networks can effectively support the completion of the rating prediction task. However, existing neural-topic methods consider only the use of topic information corresponding to current users and items without neighbors, whereas existing memory-based neighborhood approaches are inappropriate for the direct modeling of neighbors with topics. To address the limitations, we argue that memory networks have the ability to organize neighbors with corresponding topics well and can provide a general solution to this problem. To confirm our hypothesis, we propose two approaches. One is an augmented memory network to couple with and enhance existing neural-topic models. The other is a symmetric memory network activated by a memory reorganization mechanism, which is a compact and generalized method for rating prediction. The experimental results demonstrate the effectiveness of the memory reorganization mechanism and show that the two proposed methods have advantages over existing state-of-the-art topic modeling approaches.


I. INTRODUCTION
The fusion of topic modeling and matrix factorization (MF), called Collaborative Topic Modeling (CTM) [1], is an important strategy in recommendation research. The purpose of CTM is to make full use of the semantic information on topic modeling and the learning ability of factorization models [2], [3]. Compared to factorization models, neural networks have a stronger ability to learn and generalize [4], [5]. Therefore, the fusion of neural networks and topic models, called Neural-Topic Collaborative Filtering (NTCF), can fully utilize and optimize topic models by using neural network technology [6]- [8]. For example, a neural network can simulate topic modeling to generate a review semantic vector [7].
The associate editor coordinating the review of this manuscript and approving it for publication was Adnan Abid.
In the application scenario, however, we may already have considerable semantic information obtained by topic modeling, and we do not need to let the neural network learn the semantic information again. For instance, some models can directly utilize existing topic vectors extracted from user and item reviews to accomplish recommendation tasks [8]. However, this way of modeling existing topic information is not sufficient because it considers only the topics of the current user and the current item; while ignoring the rich topic information provided by user neighbors and item neighbors. The neighborhood-based factorization model [9] and the neural network [10] both fuse user-item interactions with neighbors to provide recommendations. An advanced method of neighborhood modeling [11], [12] is to integrate neighbor information into the memory network [13], [14], which can enhance collaborative filtering (CF) performance. Both neighborhood and non-neighborhood methods provide solutions for NTCF. However, these methods still have the following shortcomings: 1) The existing memory-augmented methods are not suitable for the direct modeling of neighbors with their corresponding topics. Because the topics are pre-trained and fixed semantic information, existing memory processing methods do not provide a strategy for using fixed neighbor information (topics) to assist in learning dynamic neighbor features.
2) Furthermore, there are multiple relationships among users, items, neighbors, and topics. Modeling directly from the perspective of NTCF will increase the complexity of the model, and existing methods rarely consider capturing the relationships uniformly to reduce the model complexity.
According to these limitations, we argue that: • By establishing the correspondence between topics and neighbors, memory networks have the ability to learn neighbor features with the help of the fixed topic information.
• Based on the correspondence between topics and neighbors, we can uniformly model users, items, neighbors, and their corresponding topics by effectively organizing the memory network. To address the above limitations and to confirm our hypothesis, this paper makes the following contributions: 1) To verify the ability of memory networks to organize neighbors with corresponding topics, we propose an approach called keeping neighbors in memory (KNM) to couple with and enhance existing neural-topic models.
2) To uniformly model users, items, neighbors and topics, we merge users and items into neighbor memory to obtain an entity memory, and then fuse their corresponding topics to form a topic memory, thereby establishing a symmetric memory structure.
3) Based on the symmetric memory structure, we propose an interactive self-attention mechanism to adaptively reorganize the entity memory and the topic memory. The reorganizing memory (ROM) approach is a compact and general multiple-hop memory network designed for the rating prediction task. 4) To prove the effectiveness of KNM and ROM, we first investigate several factors influencing the two models in our experiments. Then, we conduct ablation study on the reorganizing memory mechanism and finally carry out method comparison to confirm that both KNM and ROM outperform existing state-of-the-art topic modeling approaches.
The remainder of the paper is organized as follows. In Section II, research work relevant to our approach is discussed. Then, we introduce our approaches in detail in Section III and evaluate them in Section IV. Finally, in Section V, we summarize our work and provide a future vision.

II. RELATED WORK
A. COLLABORATIVE TOPIC MODELING CTM [1] is an important recommendation strategy that mainly uses MF with topic modeling to complete a recommendation task. MF [15]- [17] is a representative approach in CF to obtain accurate recommendations, while topic modeling can assist recommendation by extracting interesting topics from relevant contents [18]. One typical approach is the Hidden Factors and Hidden Topics (HFT) model [2], which bridges the gap between the latent vectors in MF models and the probabilistic vectors obtained from topic models. In contrast, TopicMF [3] jointly considers user ratings and topics from unstructured reviews by designing a transform function to align latent factors with the corresponding topic distribution parameters. Ratings Meet Reviews (RMR) [19] seamlessly combines topic models and factorization methods based on a symmetric probabilistic generative approach. In addition, the Rating-Boosted Latent Topics (RBLT) model [20] learns user preference distributions, item recommendability distributions, and latent rating factors in a shared topic space to subsequently introduce them into a MF framework for recommendation. Reference [21] uses a topic model to extract explainable aspects and learn the aspect importance/weights associated with latent factors for rating prediction tasks.

B. NEURAL-TOPIC COLLABORATIVE FILTERING
There is an inextricable relationship between neural networks and MF models [4], [22]. For example, Neural Collaborative Filtering (NCF) [23] generalizes MF from the perspective of neural networks to achieve CF. A variant of NCF is to model neural networks from a probabilistic perspective [24]. Another variant is the fusion of neural networks and topic models to accomplish a recommendation task [7], [8], which we call NTCF. NTCF can fully utilize and interpret a topic model from the perspective of neural networks [6] and try to optimize the topic model using neural network technology. For instance, the Transformational Neural Network (TransNet) [7] introduces a transformation layer to transform the combination of user and item reviews into a target review approximation that is similar to the topic of that target review. In contrast, the Adaptive Aspect Attention-based Neural Collaborative Filtering (A 3 NCF) [8] model seamlessly fuses topic vectors extracted from user and item reviews into a neural network and utilizes topic vectors from multiple levels. However, this method does not consider how to make full use of neighbors' topic vectors to better express user preferences.

C. NEIGHBOR AND MEMORY NETWORK
Nearest neighbor technology has been widely used in MF models [9], [15], [25] to mine user preferences. The nearest neighbor technique [26] can be combined with neural networks not only to select a candidate list of recommendation items but also, more importantly, to enhance performance [27]. Specifically, the Neighborhood-based Neural Collaborative Filtering model (NNCF) [10] finds a neighbor through the user-item interaction information, and then fuses this interaction with neighbors through a Multi-Layer Perceptron (MLP) network to provide recommendations. In the literature [28], users and their neighbors are used as input, and a dynamic memory network [14], [29] is used to learn personalized item representations in order to complete an opinion recommendation task. This approach proves that combining neighbors with memory networks is effective. A typical example is the Collaborative Memory Network (CMN) [12], which captures the complex interactions between users and items by proposing a memory network based on neighborhood information. However, existing memory-augmented CF methods [11], [12] are not modeled entirely from a memory perspective, while others require additional knowledge to enhance performance [30]. In contrast, we propose a memory-enhanced approach and a unified memory modeling framework and compare them with each other.

A. KEEPING NEIGHBORS IN MEMORY FOR NEURAL-TOPIC MODELING
We propose the KNM approach to enhance existing neural-topic modeling techniques by augmenting memory. The basic notations of KNM are listed as follows: • θ u -the topic vector of user u, • ϕ i -the topic vector of item i, • p u -the embedding vector of user u, • q i -the embedding vector of item i, • {η 1 , η 2 , · · · , η N } -the N neighbor vectors that are kept in memory, and • {t 1 , t 2 , · · · , t N } -the N topic vectors corresponding to N neighbors that are also kept in memory. Here, θ u and ϕ i denote user u's preferences and the characteristics of item i in the form of topic vectors, respectively. The topic vectors represent different aspects that can be extracted from reviews [8]. In addition to topic models, other models (e.g., knowledge-based methods [30]) can extract the user's preference vector and the item's feature vector. Therefore, the approach we propose is not limited to topic models, but can be extended to a general memory-enhanced CF technique that uses prior knowledge. In particular, the prior knowledge can come from neighbors. We first formally define the concept of neighbors as follow: • user neighbors -different users who have rated the same item, • item neighbors -different items that have been rated by the same user. Hence, there are two types of neighbors: users and items. For convenience of discussion, we use user neighbors to describe the proposed approach. Similar to users and items, a neighbor has a corresponding topic vector in addition to its own vector. The difference is that the neighbor vectors and the corresponding topic vectors are uniformly organized and stored in memory. Figure 1 shows the general structure of KNM as follows.
The main idea of KNM is to expand existing state-ofthe-art neural-topic models by storing a large number of useful neighbors and their corresponding topic information through memory. On the right side of Figure 1 is an advanced NTCF model that uses embedding vectors {p u , q i } and topic vectors {θ u , ϕ i } of both the user and the item as input. NTCF can be any specific advanced model, such as A 3 NCF [8].
On the left hand side is the main part of KNM, which is connected to NTCF through user input information. Specifically, element-wise multiplication is performed on the user embedding p u and the user topic θ u ; then, the obtained vector and the neighbor vectors are vertically concatenated. The concatenated vector can be used to learn the relationship between the current user and the neighbors. Considering that KNM can also be implemented by item neighbors, we refer to the current user or current item as the current entity. With the user as the current entity, we define the neighbor-entity relation as follows: Notations W A and B A in Equation 1 are the weight and bias of a full connection (FC) layer, respectively. A n refers to the relation between the n-th neighbor and the current entity. Because the n-th neighbor η n has a one-to-one correspondence with the n-th topic t n in memory, A n can be used as a connection between the current entity and the neighbor topic. To make effective use of this connection, we first use the form of factorization to directly represent the association between the neighbor and its corresponding topic, i.e., η T n · t n . η n is a dynamic memory (trainable) whereas t n is a static memory (untrainable), because each neighbor's topic is fixed in memory after topic model learning occurs. According to the nature of the two kinds of memory, we formalize the entity-neighbor-topic ternary relationship as A n · η T n · t n . The actual connection of the ternary relationship is performed by η n because A n is a function of η n . A n can also be regarded as an attention weight [31]- [34] that adopts the ideas of self-attention [33] and dot-product attention [32] but is not limited to them. The ternary relationship integrates the information of the n-th neighbor with its topic and associates them with the current entity. Neural Topic-Neighbor Interaction Modeling in Figure 1 fuses the ternary relationships generated by the N neighbors to obtain the contributions of the neighbors, which are defined as follows: The fusion of N neighbors (F N n=1 ) is actually an N to 1 dimension reduction process. The original end-to-end memory network [14] uses matrix multiplication for this reduction step, however, this mechanism for the Q&A system is not suitable for neighbor fusion because neighbor information may be offset by each other. Instead, we employ LSTM [35] to complete the dimension reduction process, because the LSTM structure [36], [37] can help us automatically integrate neighbors one by one to reduce information loss. Specifically, we sort A n into ascending order and then input the corresponding neighbors one by one into LSTM to complete the information fusion. Then, the contribution function W C X + B C (i.e., a FC layer) takes the fusion information as input to output neighbor contribution C η . Letting the CF contribution of the NTCF part be C F , we concatenate the contributions of the two parts and then perform final prediction as follows.
Here,r ui can be the predicted value of rating prediction or item recommendation task, depending on different loss functions. In experiments, we use the mean squared error to complete the rating prediction task to evaluate various models. The KNM part is an important complement to the NTCF part and is used to enhance the performance of the original neural-topic model. Moreover, the KNM part is loosely coupled to the NTCF part, which allows it to fuse with most of the neural topic models to output the final result. Therefore, KNM is compatible with the existing NTCF approaches. Similar to KNM, existing memory-augmented approaches [11], [12] can be divided into a memory processing part and a CF part, furthermore, this category of memory augmentation can be improved by a pure memory structure. In Section III-B, we perform memory reorganization on the rich neighbor-topic information and further propose an approach that does not require an additional NTCF part to implement neural-topic modeling independently.

B. REORGANIZING MEMORY
In KNM, we use only the current entity (user) to interact with neighbors, while another entity (item) is modeled in the NTCF part. In ROM, however, we would like to perform modeling uniformly from the perspective of memory processing, which is different from KNM based on memory augmentation. The challenge lies in that users, items, and neighbors all have their corresponding topics; if users, items and neighbors are distinguished from the perspective of collaborative filtering as usual, it is obviously impossible to uniformly model them. Further, directly modeling all entities, neighbors and their corresponding topics complicates the model and is contrary to our idea of implementing a simple memory-based neighbor-topic approach. To meet these challenges, we reorganize the neighbor and topic memories to establish a symmetric memory structure, thereby implementing a two-way memory interaction mechanism. This symmetric memory architecture can effectively reduce redundant interactions between entities and topics, as shown in Figure 2.
To implement memory reorganization, we first integrate the two entities and their corresponding topics into memory as follows: where Con means concatenating operation. The unified concatenating of entities and topics into neighbor memory and topic memory is the first step of reorganizing memory, allowing us to represent the unified entity memory and topic memory as ξ N ui and τ N ui , respectively. This operation ensures that there is a one-to-one correspondence between entities and topics in the same positions in ξ N ui and τ N ui . Based on this symmetric structure, we propose an interactive attention mechanism to reorganize the entity memory and the topic memory. Interactive attention is an improvement of self-attention mechanism adaptive to memory organization. The self-attention mechanism is based on the scaled dot-product attention [33], which is defined as follows: where Q, K, and V represent queries, keys, and values, respectively, and √ d is a scale factor that changes with VOLUME 8, 2020 the input dimension. From the memory perspective, given a memory element M as input, the self-attention operation transforms M into queries, keys, and values through linear projection and feeds them to the attention layer: where W Q , W K , W V ∈ R d×d are projection matrices. In this way, the relationships between memory elements are organized and expressed as attention. However, the self-attention only captures the relationship between the same memory elements, and what we need is to better organize memory through the connection between entities and topics. Therefore, we propose a memory reorganization mechanism that uses entities and topics as queries for each other to generate attention as follows: Here, TopicM is the entity-attentive topic memory and EntityM denotes the topic-attentive entity memory. In the experiments, we perform ablation study to investigate the usefulness of memory reorganization. Specifically, we use self-attention to organize memory instead of our reorganization mechanism and name this method ROM − . Because the reorganized elements in two memories are presented in the form of multi-head attention neighbors, they can be aggregated by the LSTM in the same way as in KNM. Moreover, we perform layer normalization L Norm technology suitable for the attention mechanism to cooperate with LSTM to obtain the two memory aggregations as follows: Then, modeling the interaction of the aggregated topic T ui and entity E ui can generate user preferences. We perform element-wise multiplication between T ui and E ui and enter them into a fully connected layer to interact as follows: where W ROM and B ROM are the weight and bias. The fully connected layer can be replaced by a simple or complex neural network, depending on the application. The final rating predictionr ui denotes the user preference.
To make ROM more scalable, we extend it to a multiple hop [13], [14] version and present implementation in detail in Section III-C.

C. IMPLEMENTATION OF ROM WITH MULTIPLE HOPS
The neighbor memory in ROM is trainable, while the topic memory is not trainable. Consequently, the main difficulty lies in indirectly updating the topic memory by training the neighbor memory. In the previous section, we have obtained the topic aggregation T ui , which is the bridge between topics and entities. Therefore, in a multiple-hop structure ROM, we can make full use of this relation to update the topic memory as follows: where n indicates the n-th topic or neighbor in the memory, and h is the number of the hop, starting from 1. W h is a weight that transforms the dimension of T h ui into the same dimension of η h n . This memory update mechanism associates static topics with dynamic neighbors, causing the topic to change as the number of hops increases. So far, we have introduced the theory of the ROM framework, and its detailed implementation is shown in Algorithm 1, which is implemented by Keras. In terms of memory updates, we reduce the amount of computation by updating only the topic memory, which makes up for the inability to train the topic memory. To this end, in the experiment, we test this approach by setting different combinations of parameters, such as the number of neighbors and the number of hops, and compare it with KNM and the state-of-the-art neural-topic modeling approach.

IV. EXPERIMENTS
In this section, we conduct experiments to answer the following research questions: RQ1: How to set the memory capacity of KNM and ROM? RQ2: How does the neighbor type and the hop number influence the performance of the two models?
RQ3: Does ROM's reorganizing memory mechanism make it superior to current state-of-the-art models?

A. DATASETS AND SETTINGS
To evaluate our approaches, we carry out various experiments on the Amazon Product Review Data 1 provided by [2] to accomplish the rating prediction task because this dataset is widely adopted in many topic oriented methods for experimental evaluation [2], [7], [8], [19], [20]. The items in the Amazon dataset are categorized, and a rating score ranging from 1 to 5 is used to assess a user's preference for these items. We adopt the same five categories of items as used in A 3 NCF [8], which are Patio_Lawn_and_Garden (Garden), Grocery_and_Gourmet_Food (Grocery), Baby (Baby), Sports_and_Outdoors (Sports), and Home_and_Kitchen (Home & Kitchen) shown in Table 1.
For objective evaluation, the basic experimental setup, including the preprocessing and partitioning of the dataset, is also consistent with the A 3 NCF method. For instance, Adam is employed as the optimizer, and the ratio of training, validation, and testing sets is 8:1:1 for each user [2], [7], [19]. In particular, we use the 5-dimensional topic vectors extracted by the topic model in A 3 NCF. Therefore, the factor dimensions in KNM and ROM are also set to 5 to ensure consistency with the dimension of the topic vector for the fusion of the two. Since dataset Garden is relatively small, we set the learning rates of A 3 NCF, KNM, and ROM on Garden to 0.001, while the learning rates of the three methods on the larger datasets are consistently set to 0.00005.
To verify our proposed KNM and ROM approaches from various aspects, we first investigate the impact of memory capacity on model performance to answer RQ1 in [η 1 , η 2 , · · · , η N ] h -N neighbors in memory of hop h.
[t 1 , t 2 , · · · , t N ] h -N topics for neighbors in memory of hop h. Output: r ui -the rating prediction of ROM. Initialize Entity and Get Topic

22:
previous entities 23:   Section IV-B. Then, in Sections IV-C and IV-D, we consider the effects of different neighbor types and different hop numbers on the performance of the two models for RQ2.
Finally, to answer RQ3 in Section IV-E, we conduct ablation study and compare multiple topic oriented approaches with our memory-based models to show the advantages of the two neighbor memory modeling methods. All experimental results are measured in terms of MAE and RMSE, as they are common metrics for rating prediction tasks.

B. MEMORY CAPACITY (RQ1)
Memory capacity is an important factor to consider for memory-based neural networks. Since the KNM family is characterized by putting neighbors in memory, memory capacity here refers to the number of neighbors. Intuitively, memory with a small capacity cannot load enough neighbor information to affect performance, while excessively large memory wastes storage space and increases computational complexity. Therefore, a medium-sized dataset is suitable for use in evaluating the impact of memory capacity on performance, and both Grocery (13, 979 users with 149, 434 ratings) and Baby (17, 177 users with 158, 311 ratings) datasets satisfy this condition. We set the range of memory capacity (i.e., the number of neighbors) from 1 to 10 and train KNM and ROM according to the initial learning rate of 0.00005 until convergence. The performances on the two datasets are shown in Figure 3.
The relative optimal performance on the Baby dataset occurs in the case of a memory capacity of 7, while the relative optimal performance on the Grocery data set corresponds to a memory capacity of 8. Both the MAE and RMSE curves show a trend that is higher at both ends and lower in the middle; this phenomenon is consistent with the intuition that memory with too small or too large a capacity has a negative impact on performance. However, these two curves are not smooth, and one of the important reasons is the choice of Top-N neighbors. For example, the similarity between the sixth (Top-6) neighbor and the current user may be higher than the similarity between the seventh (Top-7) neighbor and the current user, but the sixth neighbor does not necessarily contribute more to the prediction of current user preferences than the seventh neighbor. In other words, from the first to the tenth neighbors, the similarity can be monotonically decreasing; however, the contribution of neighbors to the preference prediction is not monotonically decreasing. This suggests that in addition to considering memory capacity, we must also consider the impact of other factors. In the next section, we consider the impact of both memory capacity and neighbor type on model performance.

C. NEIGHBOR TYPE (RQ2)
According to the discussion in Section III-A, there are two types of neighbors: users and items. In the previous sections, we introduced KNM and ROM with user neighbors as examples. For KNM, to implement a version of the item's neighbor memory, we need only treat η n as the n-th neighbor of the item and t n as the topic of the n-th neighbor. For ROM, implementing a version of the item's neighbor memory only requires replacing the user neighbors in both the entity memory and the topic memory with the item neighbors accordingly. Evaluating the impact of neighbor type on performance requires first determining the size of the memory capacity (that is, the number of neighbors). The experimental results in Section IV-B show that the ideal performance cannot be achieved if the memory capacity is too small or too large. Therefore, in this experiment, we empirically set a relatively small memory with a capacity of 3 and a relatively large memory with a capacity of 6 to help evaluate the impact of neighbor type. Abbreviating the user as U and the item as I, we use memory size and the UI notation to name the different versions of our approaches in order to distinguish them. For example, KNM-3I represents a KNM model with a memory capacity of 3 and an item neighbor type, while ROM-6U indicates a ROM method with a memory capacity of 6 and a user neighbor type. We experiment with all possible versions of KNM and ROM using the previous default settings, and the results of the experiments are listed in Table 2 and Table 3 according to this naming convention.   In terms of overall performance, ROM is superior to KNM because ROM does not require additional NCF methods and it makes full use of the neighbor's effective information by reorganizing the symmetric memories. For the small Garden dataset, the best performance is achieved by storing user neighbors and topics with a memory of size three. Larger data sets achieve optimal performance with a memory size of six, except that the Sports dataset uses the user neighbor ROM with a memory capacity of three to achieve optimal performances. ROM-6I achieves the best results for Grocery and Baby on both MAE and RMSE, while ROM-6U achieves the best performance for Home & Kitchen when evaluated by the two metrics. The experimental results show that both the user neighbor approaches and the item neighbor models can contribute to performances. In practical applications, in addition to considering memory capacity and neighbor types, a more generalized memory network must also consider the number of hops, which we discuss in the next section.

D. MULTIPLE HOPS (RQ2)
KNM is intended to complement the NTCF-type model, while ROM is a symmetric memory network that facilitates the implementation of multiple hops. Therefore, Algorithm 1 implements a multiple-hop version of ROM. The purpose of this section is to investigate the performance of the multiple-hop ROM. Following the naming convention of Section IV-C, we indicate the number of hops with the letter H and add this letter after a certain version. For example, ROM-6U-2H indicates user neighbor memory containing two hops, each with a capacity of six. We specify the number of hops from 1 to 3, and the performances on the five datasets obtained from the experiment are shown in Figure 4.
The best performing methods in the Garden, Grocery, Baby, Sports, and Home & Kitchen datasets are ROM-3U-1H, ROM-6U-2H, ROM-6U-2H, ROM-3U-2H, and ROM-6U-1H, respectively, with MAE as the metric and ROM-3U-1H, ROM-6U-2H, ROM-6I-2H, ROM-3U-1H, and ROM-6U-1H, respectively, with RMSE as the metric. Of these best-performing methods, the models use one hop and those employ two hops each account for 50%(5/10). Of the methods using one hop, 60%(3/5) needs a memory capacity of 3 and the remaining 40%(2/5) requires a memory with a capacity of 6. Among the methods using two hops, 80%(4/5) use a memory with a capacity of 6. The relationship between the hop number and the memory capacity indicates that large hop numbers require large memory capacity support to achieve better performance. Furthermore, 10%(1/10) of the models are based on item neighbors, while as many as 90%(9/10) are based on user neighbors. The experimental results show that using user neighbors is a good choice because they directly reflect the current user's preferences. In this case, using an appropriate memory capacity with the relatively suitable hop number according to different datasets generally leads to the desired performance. Thus far, we have considered all factors that can affect both the KNM and ROM methods. In Section IV-E, we compare these two approaches with other representative models and analyze the results.

E. METHOD COMPARISON (RQ3)
For method comparisons, we first simplified the memory reorganizing mechanism of ROM to the self-attention mechanism, and named the new version ROM − . The purpose of this process is to better examine the effect of memory reorganization, that is, to complete ablation study while comparing all the methods. Then, we conduct a preliminary comparison between A 3 NCF,KNM, and the two ROM approaches using the default settings in [8] and set the factor dimension to 5. The experimental results are shown in Table 4.
For ablation study, we observe that ROM performs better than ROM − on both metrics, and ROM − is superior to the other two methods. This confirms that the self-attention mechanism is effective in organizing memory. However, in this scenario, the mechanism of reorganizing memory has the best performance because it makes full use of the correspondence between entities and topics. In other words, the one-to-one correspondence between entities and topics can be correctly reflected, which is the reason and the advantage of reorganizing memory.
By considering all the methods, the 5-dimensional KNM and ROMs perform better on both MAE and RMSE than A 3 NCF. In particular, KNM and ROMs have greater advantages on the MAE metric. This preliminary experiment confirms that both KNM and ROMs perform well in the case of a low factor dimension. Then, we further carry out a complete method comparison by employing BMF [15], HFT [2], RMR [19], RBLT [20], and TransNet [7]. According to the results of the experiments from Section IV-B to Section IV-D, the KNM versions with the best performance on the five data sets {Garden, Grocery, Baby, Sports, Home & Kitchen} are {KNM-6U, KNM-4U, KNM-7U, KNM-3I, KNM-3U}, respectively. In contrast to KNM, ROMs add multiple hops (hop number ≤ 3), so the best version of ROMs on these data sets are {ROM-3U-1H, ROM-6U-2H, ROM-7U-1H, ROM-6U-2H, ROM-6U-1H}, respectively. We use the above versions of KNM and ROMs to carry out full comparison, where we adopt the settings in [8] and set the factor dimension of all comparison methods to 25. The experimental results are shown in Table 5.
In terms of overall performance, the NTCF models (TransNet, A 3 NCF) have advantages over the CTM approaches (BMF, RMR, HFT, RBLT). As a representative of the NTCF model, the 25-dimensional A 3 NCF is the best performing model of the compared methods. In contrast, KNM and ROMs outperform A 3 NCF and have the best performances on all the datasets. On the one hand, the experimental results confirm that neighbor-based memory augmentation can help the NTCF class method achieve better performance (i.e., the KNM model). On the other hand, this neighbor-based memory modeling approach can be implemented as a unified and symmetric memory network (i.e., ROM) that can achieve better performance than the pure NTCF approach and the memory-augmented NTCF approach. To summarize, KNM can store and express more neighborhood information than the existing NTCF methods, whereas ROMs can model rich neighbor information by a memory reorganizing mechanism that is independent of the NTCF approaches. Therefore, KNM and ROM are not only suitable for topic modeling but also work for other scenarios with rich neighbor information.

V. CONCLUSION AND FUTURE WORK
In this paper, we propose two methods for organizing neighbors and their corresponding topics using memory networks to study a general way to fully utilize neighbors and their related information. Information related to a neighbor is not limited to its corresponding topic; other knowledge may be associated with that neighbor. Therefore, this generalized neighborhood modeling approach can also be used in other cases. In future work, we will extend the two neighbor-topic modeling ideas to knowledge-based applications, enabling our framework to be competent for other recommendation scenarios. LIN