Trigger-Finder: A Real-Time Freezing-of-Gait Trigger Detection System Using an Instruction-Tuned Multimodal Large Language Model

Freezing of Gait (FoG) is a debilitating symptom of Parkinson’s disease (PD), often triggered by specific environmental factors. Existing studies focus solely on detecting FoG after it occurs. This reactive approach fails to prevent episodes or mitigate associated risks ahead of time. To address this limitation, we propose Trigger-Finder, a novel real-time FoG trigger detection system powered by a multimodal large language model (MLLM). To adapt MLLM for FoG trigger detection, we instruction-tune an open-source MLLM using a newly curated dataset comprising five common FoG triggers identified by domain experts: floor pattern changes, tight turns, dual-tasking, narrow passages, and distractions. For ensuring real-time detection, we offload the MLLM inference tasks to an edge server. We evaluate Trigger-Finder on a new trigger benchmark dataset collected from open environments. The evaluation results show that our approach significantly outperforms the baseline methods in both accuracy and inference speed. It identifies FoG triggers in approximately 0.8 seconds, enabling real-time detection in open environments. Additionally, two case studies illustrate our method enhances both accuracy and interpretability, demonstrating its potential to prevent or mitigate FoG in advance.CCS Concepts• Computer systems organization → Real-time system architecture.