Table Tennis Stroke Classification from Game Videos Using 3D Human Keypoints

In this paper, we propose a classification method for table tennis strokes using player’s 3D joint coordinates. In existing studies on stroke classification, classification is performed based on videos taken by a camera installed on a table tennis table or data obtained by attaching an inertial sensor to a player. However, in these existing methods, sensors and cameras interfere with the game, and it is difficult to adapt to the game videos. Therefore, in this paper, we classify strokes from videos that can be taken during games under more practical conditions. For the classification, we use the player’s 3D joint coordinates as input to classify the game video. In our method, we use deep learning to learn two kinds of information, the recorded video and the player’s joint coordinates obtained by 3D pose estimation and perform classification. As a result of the experiment, in the classification of the rally video dataset, the accuracy is improved by 4.2~15.5% in the validation data, which is the stroke of the learned player, and by 1.8~15.0% in the test data, which is the stroke of the unlearned player, compared with the existing method which input the videos and 2D joint coordinates.