To Watch or Not to Watch: Video Summarization with Explicit Duplicate Elimination

Video summarization is the process in which we extract key frames to form a storyboard representing the content of a video sequence. When asked to select images representing a video sequence, users attempt to find images related to the subject of the video. For the computer, such semantic analysis is a very hard problem. Still, it is possible to objectively identify the individual scenes from a sequence to extract a single key frame summarizing each scene. We propose a recursive method that broadly identifies shots in the sequence and cluster them in possible scenes via global image features. Shot boundary are usually blurry and faded and are removed along with the neighboring frames from the summary construction. A key frame is selected to represent each cluster before recursively analyzing its content. The key frames included in the hierarchical representation are analyzed for redundancy using local SURF features \cite{surf}. Key points enable the recognition of similar scene elements in the key frames, efficiently eliminating redundant information. The whole method is interactive: the user select the number of key frames to extract and the recursion depths to explore. The flexibility gained from the hierarchical representation allows the user to explore the key frames with a variable level of detail in an intuitive manner.