Rate-coverage analysis and optimization for joint audio-video multimedia retrieval

In this work, we consider the problem of automatic content retrieval (ACR) using joint audio-video fingerprints. We focus on how to balance the query accuracy and the size of fingerprint, and how to allocate the fingerprint bits to video and audio frames to maximize the query accuracy. By introducing a novel concept called coverage, which is highly correlated to the query accuracy, we are able to construct a rate-coverage model and formulate the joint audio-video fingerprint bit rate allocation into a dynamic programming optimization problem. Our experimental results demonstrate that, compared to existing approaches, our method improves the retrieval accuracy by up to 25% while using 60% of the original fingerprint bit rate.