Applying FOSSE Approach to Image Data for Evaluating Game Situations

In this paper, we propose to apply a FOrward Simulation for Situation Evaluation (FOSSE) approach to image data. A FOSSE approach was proposed for evaluating situation of a soccer game in our previous work. It consists of two parts: Forward simulation part and evaluate situation part. At first, the FOSSE approach simulates the future situation by a simulation model before evaluating the current situation. Secondly, the approach evaluates the current situation by a evaluation model with simulated future information. In the FOSSE approach, the future game situations are generated by what is called forward simulation. The first FOSSE approach used numerical information such as the position of the ball and the players. Applying the FOSSE approach to image data is a difficult task because forward simulation of image data is not straight forward. Therefore, in this paper, we focus on the forward simulation of the image data and investigate the suitable image simulation model for the FOSSE approach. As the evaluation model, we employ a 3D Convolutional Neural Network. First, we show that the evaluation performance can be increased by using successive frames of field situations through a series of preliminary computational experiments. Especially, the effectiveness of using future frame information rather than using past frame information is shown. Then, we present FOSSE approach with image data where both the current and the future frame information of game situations are used to evaluate the current game situation. Computational experiments are conducted to investigate the effective forward simulation model for our FOSSE approach.