Scalable user intent mining using a multimodal Restricted Boltzmann Machine

Nowadays, search engines have become indispensable parts of modern human life, which create hundreds and thousands of search logs every second throughout the world. With the explosive growth of online information, a key issue for web search service is to better understand user's need through the short search query to match the user's preference as much as possible. However, due to the lack of the personal information in some scenario and the huge calculation when seeking for relevant user group, personalized search becomes a quite a challenging problem. In this work, we propose a novel scalable framework based on multimodal Restricted Boltzmann Machine (RBM) to do the user intent mining and prediction. This scalable framework works in an unsupervised manner, and is flexible to various situations regardless of the amount of individual information, in other words, it can handles scenarios without personal history information or limited personal history information, the more individual data the better accuracy of user intent prediction and more capable to reflect the individual's interests changing. The framework outputs a binary representation for each query log, thus to some extent, could solve data sparsity problem and reduce the computation complexity when looking for users with similar interests. The experiment results shown that, the model can learn reasonable user intent category during the learning procedure, according to the qualitative analysis of the top ranked context and websites for each class. And it can get a competitive performance when no individual data is offered. Moreover, by offering more individual data (10 history queries), the overall performance improves up to 10% of precision.