Singing Melody Extraction Based on Joint Network with Res-CBAM

Melody extraction from polyphonic music is a tough task in music information retrieval. Current deep learning-based algorithms usually ignore the importance of attention information. The attention mechanism makes the network focus on the most relevant pitch features directly. We proposed a melody extraction algorithm based on the joint network with Residual Convolutional Block Attention Module (Res-CBAM). During the preprocessing of the music signal, the input data were represented by the pitch salience function instead of spectral analysis only. The salience function is computed by the summation of harmonic energy. Furthermore, we proposed a Res-CBAM module that combined CBAM with residual learning. We added it into a joint network for dual tasks of singing voice detection and pitch estimation. Experiments on various datasets demonstrate how the Res-CBAM and the salience data representation improve the melody extraction performance. Our system achieves a high overall accuracy of 81% on the MIREX05 dataset.