A cross-dimension indexing structure for query-by-singing

This paper proposes a novel cross-dimension indexing structure for the query by singing (QBS) via singing segment input. Content-Based Song Retrieval (CBSR) is a branch of Content-Based Audio Retrieval (CBAR). Query by Humming/Singing of CBMR (Content-Based Music Retrieval)/CBSR offers a totally new retrieval method by searching the desired music or song based on the users humming or singing segments via microphone and then real-time retrieval the intended song from a large or web-based multimedia database on the internet quickly. The base of our study on which the user's acoustic input and separated and subtracted melody features, pitch features and lyrics or vocalization features of lyrics into vector index sets database. First, we consider these features among melody, pitch, lyrics or lyrics vocalization. Second, we design a cross-dimension indexing (CDIndex) to scripts these relationships. Third, from 1000 candidate for songs, using vocalization feature of lyrics value filters out 70% unlikely candidates. And then compares the melody feature of query singing segment input with the remaining 30% candidates in a detailed manner, at last search related picture. The output of QBS will be a ranked song list with lyrics, singer and picture. The CDIndex is given in the paper to descript how to builds the cross-linking-index each other among melody, pitch, keyword of lyrics, and picture. The proposed CDIndex can be extended applied to cross-media information retrieval system in digital multimedia database or over the Internet. It will improve the user's experience by applying this new song retrieval method.