Video-Based Human-Object Interaction Analysis for Patient Behavioral Monitoring

Video-based patient behavior analysis offers deep insight into patient activities, adherence, risk assessment, and clinical decision-making. By integrating computer vision and machine learning, healthcare providers can monitor and analyze patient behaviors, improving quality of care, recovery progress, and treatment. In this paper, we propose a method to systemati- cally monitor patient behavior by analyzing interactions between a patient (in various postures) and clinically significant objects. Our methodology includes a CNN-based person/object detection & tracking, posture classification, ambulation assessment, depth estimation, and interaction analysis. A customized clinically relevant image dataset is constructed for training and validation. Our platform has achieved promising performance, i.e. an average F1 score of 98% for posture classification and an average accuracy of 90% for interaction detection.