Attention Mixture based Multi-scale Transformer for Multi-behavior Sequential Recommendation

Sequential recommendation aims to predict the next item based on historical user interactions, which is crucial for online e-commerce platforms. Most existing methods rely on singular type of interactions for sequence modeling to understand user preference, overlooking the heterogeneous behavior information between users and items. From empirical analysis, we discovered that incorporating behavioral context into learning process of user preference is effective. However, oversimplified feature approaches are limited to model performance. To this end, we propose an Attention Mixture based Multi-scale Transformer framework to address this limitation. Specifically, we devise an attention mixture module that jointly considers user-item interactions and behavioral context to capture users’ personalized multi-behavior dependencies. It allows to perform effective behavior-aware sequence modeling. Then, we incorporate the attention mixture module into a multi-scale transformer to capture the periodic patterns in multi-behavior sequences. Empirical results on three real-world e-commerce datasets demonstrate the effectiveness of the proposed method.