Multitrack Music Generation Combining Transformer and Diffusion Model

Recent advancements in generative models have made remarkable progress in music generation. However, since most existing methods focus on generating single-track music, generating multitrack music with rich attributes remains a challenging task. In this paper, we propose a novel music generation method based on the combination of the Transformer and the diffusion model. Specifically, the Transformer module is adopted to conduct structural representation learning for multitrack music, and the diffusion model is utilized to gradually denoise and generate structured music. To further enhance inter-track coherence, a track alignment loss is introduced during the generation process to enforce multitrack consistency in rhythm and dynamics, effectively resolving track coordination issues inherent in traditional diffusion models. We conduct objective experiments on evaluation metrics, namely pitch class entropy, scale consistency, and groove consistency. Compared with previous multitrack music generation methods, the proposed method performs better on these metrics.