DualRec: A Collaborative Training Framework for Device and Cloud Recommendation Models

Recommendation systems (RS) play a vital role in various domains. However, under recent data regulations like General Data Protection Regulation (GDPR), traditional RS that rely on collecting user's interaction data centrally face significant challenges. Federated learning (FL) enables collaborative model training among users while keeping their private data locally. Yet, the constrained resources of devices often limit the size of the learned model, resulting in suboptimal recommendation performance. To overcome the dilemma of data accessibility and model size, we propose DualRec, a novel collaborative training framework for device and cloud recommendation models. In DualRec, users train lightweight models on devices to harness their local private data, while a larger model is simultaneously trained on the cloud server to exploit its substantial resources. Devices and the cloud server collaboratively train their models, compensating for individual limitations of model size and data availability, enabling mutual empowerment and benefits. Specifically, we introduce an efficient aggregation mechanism for recommendation models to boost the collaborative training performance of device models. With the learned device models, we propose to generate pseudo user interaction data to train the server model. To enhance the training performance of the server model, we design an automated denoising mechanism to mitigate the negative impact of noisy samples in the generated pseudo dataset. Finally, the learned knowledge of the server model is distilled to device models for enhanced on-device recommendation performance. Extensive experiments demonstrate the superior performance of DualRec compared to state-of-the-art baselines.