Emotion-Conditioned MusicLM: Enhancing Emotional Resonance in Music Generation

Nowadays, most music generation models are limited to accepting conditions from a single modality, whether text-based or neural signal-based. For most users, text represents the most accessible and intuitive input modality. However, this often dilutes the text's emotional characteristics, misaligning the emotional depth in the final music output. This article presents comprehensive research and improvement in the text-conditioned music generation model, named Emotion-Conditioned MusicLM. Building upon the existing text-conditioned music generation model, MusicLM, ECMusicLM is designed to generate music with a deeper emotional resonance while maintaining the high quality of musical output. Our research shows that combining text and emotional elements in music generation leads to the creation of emotionally resonant music. Through the experiments, ECMusicLM showed a notable capability in capturing implicit Valence-Arousal features from text prompts, significantly enhancing the emotional depth of the generated music. This study not only pushes the boundaries of AI in artistic creation but also opens avenues for future research in multi-modal emotional synthesis.