Noncontact Measurement for Autonomic Nervous System Activity Caused by Mental Task Using Video Analysis

In recent years, robots have been increasingly used in various situations. They require emotion recognition to interact effectively. In this study, we focus on the autonomic nervous system, which is related to emotion, and the constriction and dilation of facial capillaries associated with its activity, and propose a measurement system that measures the activity of the autonomic nervous system in noncontact and real-time. The metric we propose is the ratio of green to blue in facial images (G/B ratio). Participants conducted a mental task in order to verify whether the proposed G/B ratio can measure autonomic nervous system activity. At the same time, an electrocardiogram was also measured and R-R interval (RRI) and ratio of low frequency and high frequency of RRI (LF/HF), indicators of autonomic nervous system activity, calculated from it are used to confirm whether the task had activated the sympathetic nervous system. As a result, RRI and LF/HF showed that the sympathetic nerve was dominant during the task compared with before the task, and the sympathetic nerve activity was suppressed after the task. At the same time, G/B ratio also decreased compared to the resting value (indicating sympathetic dominance) and returned to the resting value after the task. This means that G/B ratio can assess autonomic nervous system activity, indicating that our measurement system can measure autonomic nervous system activity in noncontact and real-time. When introducing this system on a daily basis, it is necessary to develop processing to remove noise associated with changes in brightness values such as changes in facial posture. By applying this technology, the accuracy of daily emotion recognition can be improved, which is expected to improve the service of domestic robots.