Sensor Event Sequence Prediction for Proactive Smart Home Support Using Autoregressive Language Model

We posit that predicting sensor event sequence (SES) in a smart home can proactively support resident activities or recognize activities that have not been completed as intended and alert the resident. To realize this application, we propose a framework to support accurate SES prediction by leveraging online activity recognition. Our framework includes a novel method of applying a GPT2-based model, which is a sentence generation model, for SES prediction by taking advantage of the property that the relationship between ongoing activity and SES patterns is similar to the relationship between topic and word sequence patterns in NLP. We evaluated our method empirically using two real-world datasets where residents perform their usual daily activities. Our experimental results show the use of the GPT2-based model significantly improves the F1 value of SES prediction from 0.461 to 0.708 compared to the state-of-the-art method, and that using ongoing activity can further improve performance to 0.837. We found that the performance of the online activity recognition model required to achieve these SES predictions was about 80%, which could be achieved using simple feature engineering and modeling.

I. Introduction

Smart home systems are equipped with sensors for monitoring residents’ activities, and with actuators for modifying the environment, such as automated lights, windows, shades, and appliances. The ability to develop smart home systems that recognize human activities of daily living (ADL) is of critical importance for applications such as health monitoring and support for the elderly. A key benefit of recognizing ADLs is that a smart home is not only able to track user activity, but also responds appropriately to the current ADL [1]. For example, the smart home can increase the brightness in the room by recognizing that the resident has started reading a book, or turn the oven off that no one is left in the kitchen after dinner. Furthermore, predicting sensor events that reflect the user’s actions enhances the usefulness of the smart home by enabling proactive assistance, such as making things ready in anticipation of the resident’s next actions or turning on an appliance that would soon be needed.