Pointing and Selecting with Facial Activity

The aim of this paper was to evaluate the use of three facial actions (i.e. frowning, raising the eyebrows, and smiling) in selecting objects on a computer screen when gaze was used for pointing. Dwell time is the most commonly used selection technique in gaze-based interaction, and thus, a dwell time of 400 ms was used as a reference selection technique. A wireless, head-mounted prototype device that carried out eye tracking and contactless, capacitive measurement of facial actions was used for the interaction task. Participants (N=16) performed point-and-select tasks with three pointing distances (i.e. 60, 120 and 240 mm) and three target sizes (i.e. 25, 30 and 40 mm). Task completion times, pointing errors and throughput values based on Fitts' law were used to compare the selection techniques. The participants also rated the techniques with subjective ratings scales. The results showed that the different techniques performed equally well in many respects. However, throughput values varied from 8.38 bits/s (raising the eyebrows) to 15.33 bits/s (smiling) and were comparable to or, in the case of smiling, better than in earlier research with similar interaction techniques. The dwell time was found to be the least accurate selection technique in terms of the magnitudes of point-and-select errors. Smiling technique was rated as more accurate to use than the frowning or the raising techniques. The results give further support for methods that combine facial behavior to eye tracking when interacting with technology.