Nursing Care Teaching System Based on Mixed Reality for Effective Caregiver-Patient Interaction

In the nursing care tasks such as assistance for transferring and walking, it is necessary to provide appropriate nursing care movements depending on factors such as the patient's pose and the degree of disability. However, for novice caregivers to practice and learn appropriate nursing care, they must practice for a long time under the guidance of skilled caregivers. To solve this problem, we propose a novel framework for a system that teaches appropriate nursing care actions according to the current situation. The realization of such a teaching system requires technology to recognize the current situation and effectively teach the interaction between the caregiver and the patient. In this article, we propose a system that integrates depth camera-based pose estimation of the patient and Mixed Reality (MR) technology to present the target motion of the patient to a caregiver. To accurately present the patient's target pose to the novice caregivers, our system displays an avatar showing the patient's ideal animation overlaid on the actual patient. Experimental results show that our system can accurately instruct the caregiver about the patient's target pose in each movement procedure.