A Contactless Measurement of Mind Activity During Conversation Using Video Analysis

In recent years, global populations have been aging, resulting in social isolation and loneliness among the elderly becoming a growing problem. Therefore, it is important to reduce social isolation and loneliness. For the elderly, conversation with others can reduce loneliness and, in particular, conversation concerning their own interests can improve cognitive functions. Having an interest in conversation and autonomic nervous system activity are closely related. By measuring autonomic nervous system activity, the degree of interest in a conversation can be measured as the amount of mind activity. As a non-contact measurement of autonomic nervous system activity, the ratio of the color green and blue in the face in a video (the G/B ratio) can quantitatively indicate the expansion and contraction of facial capillaries by using the light absorption property. In this study, a validation experiment was conducted to measure the autonomic nervous system activity of elderly people during conversations. Pulse rate — the most used current measure of autonomic nervous system activity — and facial images were measured during conversation, and a questionnaire on liveness was also administered to investigate the relationship between pulse rate, the G/B ratio, and subject’s interest in conversation. As a result, pulse rate increased with interest in the conversation, and G/B ratio decreased with interest in the conversation. Furthermore, pulse rate and the G/B ratio had a negative correlation (N=10, correlation coefficient =-0.56), suggesting that the G/B ratio can measure autonomic nervous system activity during conversation. As a future prospect, the G/B ratio will be incorporated into camera video processing in communication robots and online interview systems, contributing to a reduction of the isolation experienced by the elderly by inducing effective conversation.