Ball Motion Control in the Table Tennis Robot System Using Time-Series Deep Reinforcement Learning

One of the biggest challenges hindering a table tennis robot to play as well as a professional player is the ball’s accurate motion control, which depends on various factors such as the incoming ball’s position, linear, spin velocity and so forth. Unfortunately, some factors are almost impossible to be directly measured in real practice, such as the ball’s spin velocity, which is difficult to be estimated from vision due to the little texture on the ball’s surface. To perform accurate motion control in table tennis, this study proposes to learn a ball stroke strategy to guarantee desirable “target landing location” and the “over-net height” which are two key indicators to evaluate the quality of a stroke. To overcome the spin velocity challenge, a deep reinforcement learning (DRL) based stroke approach is developed with the spin velocity estimation capability, through which the system can predict the relative spin velocity of the ball and stroke it back accurately by iteratively learning from the robot-environment interactions. To pre-train the DRL-based strategy effectively, this paper develops a virtual table tennis playing environment, through which various simulated data can be collected. For the real table tennis robot implementation, experimental results demonstrate the superior performance of the proposed control strategy compared to that of the traditional aerodynamics-based method with an average landing error around 80mm and the landing-within-table probability higher than 70%.