Autoregressive Self-Evaluation: A Case Study of Music Generation Using Large Language Models

Autoregressive models have shown significant success in many tasks such as natural language generation and music composition. However, generic training mechanisms with off-the-shelf loss functions (e.g. cross-entropy), where not much attention is paid to the specifics of the task, do not necessarily guarantee success as different data modalities (e.g. text, visuals, music) exhibit different natures. In this study, we present a novel autoregressive self-evaluation framework to assess the performance of autoregressive models with both domain-agnostic and domain-specific metrics. We demonstrate this strategy with a case study of music generation using GPT-2 within a transfer learning paradigm. We contrast and compare the effects of fundamental parameters in autoregressive generation such as the temperature in sampling and the length of the generated sequence.