CNN-LSTM based Multimodal Models for Music Generation

In this article, we tackle the dual challenges of efficiency and quality in music generation. We aim to create a model that produces high-quality music efficiently while keeping the model lightweight and the music authenticity and creativity. This research proposes a model that combines Long Short-Term Memory (LSTM) and Convolutional Neural Network (CNN) for music generation. LSTM units capture long-term dependencies in music sequences. CNN units are included to extract musical features, enhancing the model’s grasp of structure and style. We train and valid the model on the ADL Piano MIDI datasets which is a sub set of the Lakh MIDI datasets. Finding that our CNN-LSTM model excels in music creation. It reduces computational complexity, boosting operational efficiency. This paves the way for deployment on resource-limited devices like mobiles or edge computing platforms. Combining these methods ensures the model learns form rich musical features, generating coherent, stylistically consistent pieces.