Multi-voice polyphonic music transcription using eigeninstruments

We present a model-based approach to separating and transcribing single-channel, multi-instrument polyphonic music in a semi-blind fashion. Our system extends the non-negative matrix factorization (NMF) algorithm to incorporate constraints on the basis vectors of the solution. In the context of music transcription, this allows us to encode prior knowledge about the space of possible instrument models as a parametric subspace we term ldquoeigeninstrumentsrdquo. We evaluate our algorithm on several synthetic (MIDI) recordings containing different instrument mixtures. Averaged over both sources, we achieved a frame-level accuracy of over 68% on an excerpt of Pachelbel's Canon arranged for doublebass and piano and 72% on a mixture of overlapping melodies played by flute and violin.