InfraTag: Customizable Infrastructure Interaction Activity Recognition on Resource-Constrained Hardware

Recognition of infrastructure interaction activity can provide environmental context, which is essential for achieving human interaction with ubiquitous computing environments. However, there are different types of infrastructure and user habits, and existing methods generally rely on special-purpose sensing hardware or training models in situ, which are task-specific with a high user burden for open-set recognition tasks. To address this challenge, we created InfraTag, through our proposed customizable method to conduct one-shot customization that combines generalizability and flexibility. This method utilizes prototype encoding by introducing domain knowledge to learn a generic motion posture feature embedding and with a similarity-matching design to achieve one-shot customization on resource-constrained hardware. InfraTag can be attached to infrastructures, and after one-shot learning of the new infrastructure interaction activity sample during the initial deployment, ongoing recognition can be well performed. For example, we apply InfraTag to in-home infrastructures, such as doors, windows, waterpipes, etc., that can augment everyday experiences with context-aware functionality. Overall, our work demonstrates a more general approach to recognizing infrastructure interaction activities in ubiquitous scenarios, and its resource-efficient and easy-to-deploy make it a more feasible option for future consumer devices.