Software Architecture for Automating Cognitive Science Eye-Tracking Data Analysis and Object Annotation

The advancement of wearable eye-tracking technology enables cognitive researchers to capture vast amounts of eye gaze information while participants are completing specific tasks without restrictions on their movement. However, while eye trackers can overlay a gaze indicator on the scene video, identifying the specific objects being looked at and analyzing the resulting dataset are accomplished mostly by manual annotation. This method is a cost-prohibitive and time-consuming approach that is prone to human error. Such analytic difficulty limits researchers' ability to data mine the information efficiently, ultimately restricting the number of scenarios that can feasibly be conducted within budget. Here, the first fully automated solution for eye-tracking data analysis is presented, which eliminates the need for manual annotation. The proposed software architecture, gaze to object classification (GoC), processes the gaze-overlaid video from commercially available wearable eye trackers, recognizes and classifies the specific object a user is focusing on and calculates the gaze duration time. GoC utilizes an image cross-correlation method to locate the gaze indicator and an image similarity measurement to support faster processing. The presented system has been successfully adopted by cognitive psychologists. GoC's exceptional performance in analyzing a case study spanning over 50 h of mobile eye-tracking is presented. The accuracy and a cost-analysis comparison between GoC and state-of-the-art manual annotation software are provided. GoC has game-changing potential for increasing the ecological validity of using eye-tracking technology in cognitive research.