TDT: Two-Stream Decoupled Transformer for Text-Driven Human Animation Generation

The automatic generation of high-quality human animations in a controllable and natural manner has always been a goal pursued by experts in the field of animation. Text- driven human animation generation offers user-friendly and versatile application scenarios. In this paper, we propose a two-stream decoupled Transformer network, TDT, for extracting motion semantic information from the text modality and generating human animations based on it. Our contributions are: (1) The text is highly decoupled in its description of trajectories and motions, utilizing a two-stream Transformer network to extract trajectory features and motion features separately. (2) We employ vector orthogonalization to constrain the motion feature vector and the trajectory feature vector, promoting the decoupling of the motion manifold and the trajectory manifold. (3) We introduce a local loss function, the mean-variance reconstruction loss, as a complement to the global loss function, further facilitating feature fusion between the two modalities. Experimental results demonstrate that our proposed model outperforms state-of-the-art text-driven human animation generation models in terms of objective evaluation metrics. Moreover, according to visualization results, the generated motions exhibit greater consistency with the semantics described in the input text by the user and are closer to the ground truth data.