Semantic Annotation and Retrieval of Music and Sound Effects
Turnbull, D.
Barrington, L.
Torres, D.
Lanckriet, G.
Dept. of Comput. Sci. & Eng., Univ. of California at San Diego, La Jolla, CA;
This paper appears in: Audio, Speech, and Language Processing, IEEE Transactions on
Publication Date: Feb. 2008
Volume: 16,
Issue: 2
On page(s): 467-476
ISSN: 1558-7916
INSPEC Accession Number: 9977912
Digital Object Identifier: 10.1109/TASL.2007.913750
Current Version Published: 2008-01-16
Abstract
We present a computer audition system that can both annotate novel audio tracks with semantically meaningful words and retrieve relevant tracks from a database of unlabeled audio content given a text-based query. We consider the related tasks of content-based audio annotation and retrieval as one supervised multiclass, multilabel problem in which we model the joint probability of acoustic features and words. We collect a data set of 1700 human-generated annotations that describe 500 Western popular music tracks. For each word in a vocabulary, we use this data to train a Gaussian mixture model (GMM) over an audio feature space. We estimate the parameters of the model using the weighted mixture hierarchies expectation maximization algorithm. This algorithm is more scalable to large data sets and produces better density estimates than standard parameter estimation techniques. The quality of the music annotations produced by our system is comparable with the performance of humans on the same task. Our ldquoquery-by-textrdquo system can retrieve appropriate songs for a large number of musically relevant words. We also show that our audition system is general by learning a model that can annotate and retrieve sound effects.
Index
Terms
Available to subscribers and IEEE members.
References
Available to subscribers and IEEE members.
Citing Documents
Available to subscribers and IEEE members.
Cart
