An Exploration of Controllability in Symbolic Music Infilling

This study uses a transformer model to enhance the controllability of generative symbolic music models, specifically related to the infilling task. We introduce a novel Symbolic Music representation with Explicit Rest notation (SMER) encoding incorporating five basic duration types and explicit rest note tokens similar to standard music notation. We compare this approach with another event-based symbolic music encoding called “REMI” (REvamped MIDI-derived events) regarding controllability over bar-level tension and track-level texture, which refers to how musical elements such as melody and harmony are combined in a musical composition. The SMER encoding is compared with another controllable infilling model, Multi-Track Music Machine (MMM), for track-level density controllability. The findings confirm that the proposed SMER demonstrates superior controllability and generates music stylistically more similar to the original music than that generated by MMM. We propose strategies to further enhance track-level texture control by training two models, controlling each bar’s texture (SMER BAR), and predicting each bar’s texture after each bar’s generation (SMER Pre). Those two models with bar-level texture control effectively increase track-level texture control. To explore the interaction of the controllability of different controls, we thoroughly analyzed the controllability of different types and levels of texture controls. Finally, we implemented an interactive interface to facilitate interactive music composition with AI to help bridge the gap between the AI model and musicians.