Event Retrieval in Soccer Video from Coarse to Fine Based on Multi-Modal Approach

With the explosion in demand for visual information retrieval in soccer videos, many Content-Based Video Retrieval (CBVR) models were born. However these current CBVR models still remain some shortcomings such as inflexible retrieval frameworks because they are majorly based on a specific training data set and a specific language. In this paper we propose a framework which contains flexible and easy-to-use queries based on visual and audio data indexing in soccer videos. Our framework understands a soccer event which is constructed by the following visual components: shot with shot types, interest objects, and audio components. In which, an event can be retrieved from coarse to fine by finding relevant shot-sequence, after that some specific visual and audio objects are related to events. Our experiment on AFF-Suzuki Cup Tournament 2009 video data set showed better retrieval results with events such as goal, foul, shoot, corner kick, off side, red (yellow) card regarding precision and recall than the traditional systems only based on visual or audio queries.