A novel framework for video summarization based on smooth pursuit information from eye tracker data

Existing methods for video summarization fails to achieve a satisfactory result for a video with camera movement, low contrast, and significant illumination changes. To solve these problems, we propose a novel framework for video summarization based on the smooth pursuit which is the state of eye movement when a user follows a moving object in a video. First, we propose a new method to distinguish smooth pursuit from another type of gaze points, such as fixation and saccade. Later, we assign a probability score to each frame based on the smooth pursuit information. Finally, we select a set of key frames based on the probability score. To evaluate the proposed method, we implement it on Office video dataset that contains videos with camera movement/shaking and illumination changes. Experimental results show the superior performance compared to the single view results of the state-of-the-art GMM based method.