A Novel Compressive Compound Word Encoding and Independent Word Attention for Symbolic Music Generation

Symbolic music generation involves using symbolic encoding to represent music pieces as token sequences and using neural sequence models to create music by generating sequences of tokens. Symbolic encodings are primarily categorized into two types: independent word encoding and compound word encoding. Independent word encoding treats each token equally, where the model predicts one token at each timestep. Compound word encoding groups associated tokens and combines them into a super token, and places them in one timestep, where the model predicts multiple different tokens at each timestep. Previous works related to compound word encoding only use compressed representation constructed from super tokens for model learning without considering the independent tokens that consist of super tokens. To enable models to capture the dependencies between independent tokens in super tokens when using compound word encoding, we propose compressive compound word (CCP) encoding and independent word attention (IWA). The CCP reduces the number of ignore tokens in super tokens and timesteps required to encode one music piece. The IWA learns how the independent tokens in super tokens are organized. Experimental results demonstrate that IWA can improve the quality of generated music while CCP is more efficient when inference.