Temporal detection and processing of transparent overlays for video indexing and enhancement

This paper develops a theoretical model for the formation of transparent overlays and proposes a temporal algorithm to detect them independent of their degree of transparency. The proposed algorithm exploits our novel observation that the appearance of a transparent overlay results in a proportionally constant decrease in the intensity variance. In order to detect transparent regions, we first compute intensity variances about each pixel. After that, the ratios of the variances between the pixels of the consecutive frames are computed to form variance ratio images. Because the degree of transparency is unknown and may vary, we generate binary images by thresholding variance ratio images for every possible fine interval of the degree of transparency. Various morphological, textural, and contextual information are applied to every candidate binary image to detect spatial location of transparent overlays. We can also accurately detect the color and the degree of transparency of the transparent overlay so that we can remove the transparency or apply user-specific enhancement operations. We also demonstrate the application of the algorithm to video indexing and retrieval.