Music-Driven Motion Generation for Fashion Show via MP Loss

Recently, the field of music-driven motion generation has witnessed significant advancements. While a plethora of research has been dedicated to dancers and musicians, there remains a conspicuous void in its application to fashion shows. Existing literature underscores the prowess of deep neural networks (DNNs) in music-to-motion tasks, requiring minimal feature engineering. However, tailoring DNNs for fashion show motion generation poses distinct challenges. Firstly, DNNs are tasked with producing consistent and repetitive motion sequences in tandem with musical features. Secondly, there's an imperative for the generated motion to synchronize rhythmically with the musical beat. To address these intricacies, our paper pioneers motion generation techniques for fashion models, aiming to craft natural and rhythmically congruent catwalks that resonate with musical melodies. We conceptualize catwalk generation as a sequence-to-sequence machine learning paradigm. Our approach harnesses a transformer-based decoder to adeptly extract musical nuances, paired with an LSTM-based encoder that captures the nuanced interplay between sound and motion. To optimize model training, we introduce the momentum perception loss function, which integrates a multi-weight coordinate loss and a contrast loss. This ensures a harmonious rhythmical alignment between music and motion. Rigorous experiments validate the superiority of our methodology over traditional DNN techniques, yielding unparalleled motion generation for fashion models that is both fluid and rhythmically synchronized.