Motion Transfer based on Front and Back View for Rehabilitative Exercise

As a part of rehabilitation training, rehabilitation gymnastics plays an important role in restoring the patients’ bodily movement. To facilitate the evaluation of rehabilitation exercises, in this work we propose multi-view on a motion transfer framework which is based on GANs and parameterized human model. Thus we can generate virtual exercise images sampled from the front and back view of one person with the appearance features. Motion transfer methods based on 2D information can hardly deal with the ambiguity of human pose and lack of appearance features. Therefore, this work uses a 3D parametric human model to represent the human body and poses, sampling, and combining pixels from pictures token from both front and back view of a source character, to generate target images under given poses. The whole framework we used can be divided into three stages. 1) In the first stage the 3D human mesh will be reconstructed. 2) In the second stage, transformation maps of multi-view will be calculated, which can show the coordinates of required pixels for sampling and placing. 3) In the third stage, a generated adversarial network is employed to optimize the coarse result from the second stage and make it more realistic. In our experiments, the whole framework achieves good results, especially it is proved that sampling features from both the front and back view can preserve more details of appearance by comparison experiment.