Active state recognition of a person by the multimodal biological information estimated from facial image sequences

This paper investigates human activity recognition based on biological information estimated by a non-contact measurement method. A facial image sequence of a person is utilized to estimate the biological information, such as heart rate, heart rate variability and blinking. Experiments that impose physical and mental tasks are conducted in order to gather the biological information, and then experiments of recognising the person's active state are conducted to evaluate three active states of resting, physical burden and mental burden. In order to classify person's active state, machine learning approaches, such as multilayer neural network and support vector machine, are used to design recognition systems and their characteristics are investigated. In person-independent recognition, neural network using all biological information features including dynamics information can attain a recognition rate of 82% while support vector machine can achieve the recognition rate of 97%. Experimental results demonstrate that using multimodal biological information is feasible for recognising the person's active state and that support vector machine is suited for this task.