Prediction of Shoot Events by Considering Spatio-temporal Relations of Multimodal Features

In this paper, we propose a prediction method for shoot events using multimodal features. Soccer event prediction has conventionally been conducted based on statistical analysis by using an enormous quantity of tracking data. In contrast, with the development of the machine learning technology, an approach that uses advanced semantic information extracted from various data such as video, audio, and text has achieved high accuracy in many tasks of soccer analysis. Therefore, it is expected to improve the prediction performance of shoot events by cooperatively using multiple modalities instead of tracking data. Specifically, the proposed method predicts shoot events by considering spatio-temporal relations of a graph constructed based on audio and visual features extracted from soccer videos. Through the experiments, we verify the effectiveness of our method.