Estimation of Attentiveness of People Watching TV Based on Their Emotional Behaviors

We propose a method for estimating the attentiveness of viewers watching TV on the basis of their emotional behaviors. Such behaviors are usually hard to perceive and have in the past required contact-type equipment to measure them. We devised a novel attentiveness estimator that uses three visual features that are correlated with emotional behaviors: the viewer's head motion, blink interval, and eye movement. The main contribution of this paper is that the method automatically obtains these features from ordinary noncontact-type sensors. In addition, the method considers not only the viewer's emotional behaviors but also content features which affect the behaviors. The temporal window pattern in a video sequence is used to describe temporally local features. Experimental results indicate the possibility of recognizing TV viewers' attentiveness from certain visible emotional behaviors measured by common home electronics.