Fine-grained event timing detection method using quasi-high frame generation for single camera image sequence

This paper describes a method we propose that obtains event timing at sub-frame level, which is more precise than frame level, on the basis of temporal super-resolution for sports videos. Since athletes move very quickly in sports situations the required time resolution for event detection for sports motion analysis is quite fine grained. Thus, we need to detect eventos that have not been recorded even if means having to check events at every frame. The proposed method is able to generate quasi-high frame rate videos, but does so with difficulty because the videos have too high a degree of freedom. We focus on the fact that repeated motions occur in many sports, and the projection of low-dimensional feature space is obtained on the basis of these repeated motions. Interpolations in the low-dimensional feature space make it easy to provide quasi-high frame rate videos that enable event detection at sub-frame level. Experiment results verified that our method detects ball release timing from 30 fps video with mean error of less than 0.02 seconds. Moreover, subjective evaluation experiments showed that the proposed method has accuracy applicable to the sports training VR system we developed.