Segmented sequence modeling using Variational Autoencoder for landmark detection in Indian Classical Music

Musical melodies feature hierarchically ordered occurrences, with specific events standing out more than others and serving as melodic landmarks. The presence of Nyas is an essential marker in Hindustani music melodies. The presence of Nyas is essential for establishing and maintaining the framework of a Raag and defining the limits of melodic themes. Detecting Nyas segments is vital for melody segmentation, theme finding, and Raag recognition tasks. However, recognizing Nyas segments is problematic since these segments do not adhere to explicit norms involving segment length, contour features, and melodic context. This paper proposes SS-VAE, a sequence model for automatically detecting Nyas segments in Hindustani music melodies. It consists of four main steps: domain knowledge acquisition, feature extraction, sequential modeling, and segment fusion for finding the placement of Nyas boundaries. We extended the limited dataset containing only Alap sections to 67 audio music recordings containing Alap, mid, and end sections. Overall, 2269 Nyas swara occurrences and a total duration of 2.9 hours are curated. Our proposed SS-VAE model performs well across both employed datasets. Further, we show that the proposed Nyas segmentation approach improves over the peer approach.