Hierarchical Reinforcement Learning Recommendation Method based on Deep Interest Networks

With the rapid development of MOOC platforms, course recommendation has become a focal point in the research field of recommendations. Effectively modeling user preferences is a crucial task within this context. Despite the notable achievements of reinforcement learning methods in simulating user preferences, there are still existing issues with current approaches. This paper introduces a hierarchical reinforcement learning method based on deep interest networks. By incorporating an adaptive weighting unit to represent the correlation between the current course and historical courses, the method aims to better simulate user preferences. Experimental results on two real datasets demonstrate the superiority of our approach over baseline methods, and it exhibits strong generalization in other recommendation domains.

I. Introduction

Massive Open Online Course (MOOC) platforms are rapidly evolving, and open online courses are becoming increasingly popular. However, due to the vast number of courses in MOOCs, there is a need for an effective way to filter and sort through massive amounts of data, understanding user preferences for courses, and providing personalized recommendations. Course recommendation can be understood as the goal of suggesting the courses that users are most likely to choose at time t + 1 based on their course selection history up to time t. Many recommendation methods address how to capture user preferences. The Hyperedge Graph Neural Network (HGNN) [1] adopts a graph neural network for the recommendation task. It inputs the time series of embedding vectors of selected courses into a Gated Recurrent Unit (GRU) [2] model, outputting the last embedding vector as user preference. Neural Attentive Item Similarity (NAIS) [3] and Neural Attentive Session-based Recommendation (NASR) [4] simulate user preferences through attention coefficients on the history courses. Building upon this, Hierarchical Reinforcement Learning (HRL) [5] enhances the accuracy of simulating user preferences by modifying user course selection records to eliminate noisy courses. This approach eliminates the need to assign attention coefficients to each course, streamlining the process and improving accuracy in simulating user preferences.