SSGesture: Multimodal Gesture Generation Framework for Human Animation Synthesis

Technological innovations are reshaping the development of animation production. As virtual characters are increasingly used in animation creation and smart assistants, a key challenge is how to automatically generate dialogue gestures. However, current approaches often overlook a wide range of modalities and their interactions, resulting in gestures that have low contextual variation and noticeable jitter. To address these issues, we propose SSGesture, a novel diffusion-based framework that effectively captures cross-modal associations. Our three-layer attention structure enhances multimodal processing. We propose the first method to automatically resolve style conflicts through interpolation-based gesture style control, while implementing a unified unmarked style prompt structure via the PAAN layer. Our framework is practically applied in the field of intelligent virtual assistants to generate gestures in human animation synthesis and to realize various new applications. Extensive experiments and user studies have demonstrated that our proposed framework, provides substantial assistance in enhancing the efficiency of human animation production.