CPTGZ: Generating Chinese Guzheng Music From Chinese Paintings Based on Diffusion Model

In the context of rapid advancements in artificial intelligence technology, AI-powered music composition has demonstrated remarkable creative capabilities. However, no existing music generation model has been able to produce authentic waveform-level traditional Chinese music. To explore the potential of this field and address the limitations of current technologies in generating traditional Chinese music, this study introduces CPTGZ (Chinese Painting to Guzheng Music), a music generation model based on latent diffusion and Transformer architectures. CPTGZ aims to achieve automatic generation of waveform-level Guzheng music from Chinese paintings, thereby addressing the inability of existing music generation models to produce traditional Chinese music.To support the development and training of the model, we constructed a large-scale dataset of paired Chinese paintings and Guzheng music, consisting of 22,103 sample pairs. Through experimental evaluation, we found that CPTGZ exhibits excellent performance in terms of music quality and Guzheng-specific characteristics. The results demonstrate that our model can generate Chinese Guzheng music pieces highly correlated in style and semantics with the input Chinese paintings. Furthermore, the musical qualities of the generated Guzheng compositions demonstrate the characteristics of traditional Chinese music, thus validating the feasibility and effectiveness of our model.This research contributes to the field of AI-driven music generation by addressing the specific challenges of creating authentic traditional Chinese music, particularly Guzheng compositions, based on visual art inputs. The successful implementation of CPTGZ not only opens new avenues for cross-modal generation in the domain of culturally specific art forms, but also demonstrates the potential for AI to preserve and innovate within traditional art forms.