User Context Generation for Large Language Models From Mobile Sensing Data

Large language models (LLMs) exhibit remarkable capabilities in natural language understanding and generation. However, the accuracy of the inference depends deeply on the contexts of queries, especially for personal services. Abundant mobile sensing data collected by sensors embedded in smart devices can proactively capture real-time user contexts. However, raw sensing data are low-quality (e.g., existing missing data and data redundancy) and are incapable of providing accurate contexts. In this work, we present ConGen, a user context generation framework for LLMs, aiming at prompting users’ contexts through their implicit mobile sensing information. ConGen integrates two components: refined data completion and multi-granularity context compression. Specifically, the refined data completion couples data-centric feature selection by leveraging the eXplainable AI (XAI) method into the data imputation model to generate fewer but more informative features for efficient and effective context generation. Additionally, we implement multi-granularity context compression, reducing timestep- and context-level data redundancy while further elevating context quality. Experiment results show that ConGen can generate more accurate context, surpassing competitive baselines by 1.3%-8.3% in context inference on all four datasets. Moreover, context compression significantly reduces redundancy to $1/70\sim 1/40$ of the original data amount, and further improves the context accuracy. Finally, the enhanced performance of LLMs, as demonstrated by both quantitative and qualitative evaluations of prompting ConGen-generated user contexts, underscores the effectiveness of ConGen.